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This report has been prepared under Section 6 of the National Audit Act, 1983 
for presentation to the House of Commons in accordance with Section 9 of the 


Act. 
John Bourn National Audit Office 


Comptroller and Auditor General 20 September 1995 


The Comptroller and Auditor General is the head of the National Audit Office 
employing some 750 staff. He, and the NAO, are totally independent of 
Government. He certifies the accounts of all Government departments and a 
wide range of other public sector bodies; and he has statutory authority to 
report to Parliament on the economy, efficiency and effectiveness with which 
departments and other bodies have used their resources. 
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Summary and conclusions 


The world’s population is 5.6 billion and is currently growing by at least a 
billion each decade. Developing countries are facing major challenges in 
ensuring that their people can access essential health care against a 
background of financial and economic constraints, population growth and 
increasing health hazards. Many are struggling to provide accessible health care 
facilities from a health budget of less than £7 a person a year. Most countries 
are now committed to taking action that would lead to stabilising population 
growth through improved reproductive health rather than by imposing 
population control measures. 


Since April 1988 the Overseas Development Administration have spent 

£911 million on health and population assistance, mainly in Asia and Africa, 
including £393 million on emergency aid. This has represented a growing 
proportion of the Aid Programme in recent years. The pattern of health aid has 
changed in the last decade and support for reproductive health services has 
increased following the Administration’s population initiative in 1991, and will 
increase further following the adoption of the Programme of Action at the 
International Conference on Population and Development in 1994. 


The objective of the UK’s health and population aid is to enable those most in 
need to gain access to essential health care, including reproductive health 
services, in ways which make efficient use of available resources. 


The National Audit Office sought to assess how the Administration monitored 
progress on health and population projects against measurable targets and took 
remedial action where appropriate; and the success achieved in terms of project 
impact and sustainability. 


The National Audit Office examined bilateral health and population assistance 
in four countries, covering 16 projects in India, Pakistan, Kenya and Zimbabwe. 
They also examined the Administration’s contributions to multilateral 
organisations and research funding. The main findings are set out below: 


a) Good management involves close supervision of project implementation and 
prompt remedial action where necessary. Over the last four years the 
Administration have recruited 14 country-level health and population field 
managers, who have been supported over the last two years by four resource 
centres providing consultancy advice on all aspects of project design, 
implementation and impact. The Administration plan to evaluate these 
arrangements in due course to see whether they have improved aid 
efficiency and effectiveness. 
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The Administration expect project managers to report regularly on the 
extent to which objectives are being achieved and on areas requiring 
remedial action. However, they do not generally provide guidance on the 
format of progress reports required either for contracted out projects or 
those internally managed. It seemed to the National Audit Office that this led 
to some reports omitting essential detail, such as summaries of progress, 
progress against project framework or analysis of expenditure targets. 


In four projects the Administration had set broad goals (for example 
improvements in health status) without specifying a sufficiently precise 
lower-level project purpose to relate directly to project activities. In some 
other cases, purpose level objectives related to expected outputs - for 
example, the number of clinics built, rather than measures more closely 
related to purpose, such as clinic usage and treatment provided. The 
Administration indicated that the setting of appropriate objectives at these 
various levels was being given high priority in current project design. 


The Administration have not always been able to set measurable indicators 
of progress towards project objectives at the purpose level, particularly 
before projects started. Difficulties were experienced when a project's 
emphasis was on institutional development. This problem related mainly to 
earlier projects and the Administration emphasised that recent projects now 
had clearer and more quantifiable indicators against which progress could 
be measured. | 


In nine of the 16 projects examined the Administration had either not set 
measurable indicators of progress towards objectives of project purpose or 
not made them explicit. Instead they had selected indicators which were 
designed to enable impact to be assessed after project completion. Recent 
projects had more consistently set realistic indicators, with data that were 
readily available, to permit explicit monitoring of the progress of a project 
towards achieving its purpose. The Administration have initiated a major 
programme of training for the regular review of project objectives and for 
resetting of indicators during implementation. 


The Administration held up-to-date financial records for projects overall but 
in two cases did not have detailed financial breakdowns of project 
components. In those cases they therefore lacked the necessary information 
to make judgements about efficiency and effectiveness. 


The Administration seek competitive proposals wherever possible. For 
consultancy services where fees are likely to be £50,000 or less, the cost of 
competition can be taken into account as one, but not normally the only, 
factor in a decision to waive competitive tendering. For the procurement of 
goods over £100,000 the procurement agent would normally be identified by 
competition. Of the 13 contracts examined, competition was waived for 
seven. This was, in part, due to the Administration’s aim to give the British 
Council the opportunity of first refusal in certain countries and the 
Administration’s contractual arrangements giving Crown Agents exclusive 
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rights to Administration-funded procurement for certain countries. In two 
instances the Administration did not provide tenderers with detailed terms 
of reference describing the services they required. 


— 


The National Audit Office found that all the projects had been the subject of 
routine monitoring. This tended, however, to focus on inputs, due mainly to 
the need to address problems with their delivery (particularly timeliness and 
quality), and outputs. It often did not relate progress to objectives and targets 
- and hence potential impact. The Administration indicated that in larger 
projects impact was assessed through mid-term reviews but the National 
Audit Office identified only three such reviews in 13 projects examined. 


Monitoring h 


Remedial i) The Administration found it necessary to make some adjustments to the 

action design of each of the 14 projects examined to meet changing circumstances. 
There were four examples where such remedial action had been successful, 
resulting in prompt improvements. In seven other cases remedial action was 
partially successful or was effective after an extended period. In three cases, 
remedial action was not taken or was taken late. This reflects, in part, the 
difficulty in implementing prompt remedial action in collaborative 
partnership projects given the shared responsibility for implementation and 
monitoring. 


j) In the National Audit Office’s view, delays in taking remedial action were 
exacerbated because the Administration did not define appropriate 
responsibility with project managers and, in some cases, because progress 
reports had paid insufficient attention to the difficulties encountered or had 
not attached adequate priority to dealing with problems. 


— 


To assess whether a project has been successful in achieving its objectives, it 
is necessary for clear, appropriate, project-specific objectives to be set at the 
design stage and for relevant data to be collected during and after the 
project. In health projects it is difficult and often costly to make a full 
assessment of project impact on people’s health status - particularly in 
projects involving inputs from other donors, where it is impossible to 
disaggregate the Administration’s contribution from the effects of other 
factors; and in some projects impact cannot be assessed until some time 
after all inputs have been completed. In nearly half of the projects examined 
the Administration had not ensured that they had sufficient information to 
assess impact or intended impact. They have reviewed the benefits obtained 
through support to health management projects, grants to multilateral 
agencies and some reproductive health projects. 


Impact k 


1) During project design the Administration undertook economic appraisals but 
did not always give explicit consideration to whether the delivery 
mechanisms proposed were the most cost-efficient means of achieving their 
aims. They now attempt to assess the prospective impact of different 
interventions through their funded research and plan to provide relevant 
summary information to those responsible for project design. 
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economic viability of projects. They recognise that it is never easy to ensure 
the sustainability of social sector services but consider that this issue often 
needs to be addressed in a wider context than within individual projects. 
Hence for most projects examined, sustainability considerations did not 
feature prominently. The Administration explained that, nonetheless, 
discussions about sustainability of social sector services always took place 
internally and with recipient countries. The issue is also often addressed 
through donor agencies’ wider dialogue with national agencies about macro 
economic and social issues, including poverty. Much of this is not 
documented in project files. 


The Administration accept that many developing countries are unlikely to be 
able to meet the full costs of contraceptives, essential drugs and medical 
supplies. They therefore plan projects on the assumption that donors will 
continue to contribute to such supplies for many years. 


Projects are more likely to be sustainable where recipient countries 
demonstrate a strong commitment at the early stages of design; where 
well-established organisations have been able to demonstrate a previous 
record of maintaining projects; and where beneficiaries have been actively 
involved throughout. 


In two of the earlier projects examined, the Administration were unable to 
secure the full commitment of recipient institutions to project objectives or 
successfully to enhance the capacity of institutions to enable them to be more 
effective. The Administration now require an explicit commitment to agreed 
objectives at an early stage in project design. 


The Administration attempt to ensure that partner institutions comply with 
their undertaking to provide funding once the project is complete but in eight 
of the projects examined national authorities were finding it difficult to meet 
continuing costs. In half of these cases this was a result of deteriorating fiscal 
conditions. It seemed to the National Audit Office that most of the project 
proposals examined did not include sufficiently detailed assessments of the 
recurrent cost implications likely on project completion. 


The Administration have established a comprehensive process of review and 
evaluation and have considered a broad span of evidence in assessing the 
effectiveness of multilateral agencies in providing health and population aid. 


The Administration have taken steps to improve the direction of their 
funded research. They have formalised research agreements and introduced 
competition into the selection process. 


The Administration have introduced a number of remedial measures with 
the aim of directing research funding to more urgent needs and to improving 
the prospects of projects achieving their objectives. As a result the 


jf 
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Administration consider that the recently completed research work has had 
an impact on policy formulation and has enabled research skills to be built 
up in the essential health and population areas. 


Of the 14 projects that had started, the National Audit Office found that nine had 
largely achieved, or were likely to achieve, their objectives, three will partly 
achieve their objectives, and two were likely to have little impact. 


In the last four years the Administration have taken various initiatives to learn 
from past experience and improve the prospects of projects achieving their 
objectives. Changing policies for UK health and population aid have led to 
sector-focused action in specific countries to develop approaches to facilitate 
fundamental health systems development. The Administration have increased 
their emphasis on long-term partnerships with recipient countries, encouraged 
closer in-country co-operation with multilateral agencies and given less 
emphasis to discrete projects that have in the past yielded selected health status 
gains which may prove hard for the recipient to sustain. 


There have been a number of problems associated with individual projects - in 
many respects due to the serious difficulties with implementing effective health 
services in developing countries. As the Administration are working in 
independent sovereign states where they do not have direct control, it is almost 
inevitable that some projects will not be as successful as they would wish. The 
Administration have made significant progress in recent years in setting more 
realistic and measurable objectives, and in achieving them. 


The National Audit Office nevertheless consider that there are ways in which 
project success could be more strongly secured and recognise that the 
Administration are seeking to achieve this through: 


e devoting more attention, where practical and cost-effective, to ensuring at the 
outset that baseline data will be available and that targets are realistic, 
measurable and linked to project objectives; 


e focusing specific project objectives on clearly defined outcomes that are 
achievable and realistic; 


e giving more consideration to sustainability issues at the design stage - 
particularly in reproductive health and institutional strengthening projects; 


e improving the service from contracted managers by defining their role more 
clearly, for example by preparing more detailed project management 
proposals, either in-house or using consultants, and inviting contractors to 
quote fees on that basis; 


e identifying and contracting procurement agents as early as practicable and 
ensuring that they are involved in project reviews; 
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requiring all managers to submit progress reports which, as a minimum: 
show past and planned expenditure against budget for each component; 
identify whether indicators of achievement for each component need 
updating; indicate progress towards achieving outputs; indicate, with 
reasons, areas where progress is not going according to plan, including the 
extent to which project purpose is likely to be achieved; and propose 
remedial action, if appropriate, with a suggested timetable and clearly 
defined responsibilities; 


providing more guidance on monitoring and its objectives; and allocating 
specific monitoring responsibilities to nominated professional staff; 


obtaining more reliable data on progress achieved and assessing more 
systematically the potential impact of projects to see whether they are 
achieving their objectives; 


increasing their efforts to secure the commitment, and enhance the capacity, 
of recipient institutions; 


increasing their active role in seeking the reforms necessary to ensure that 
greater value for money is obtained from multilateral contributions; and 


enhancing the effect of their funded research - giving greater emphasis to 
identifying opportunities for introducing cost-effective health care, efficient 
management and effective delivery, and improving their dissemination of 
summarised results. 
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Part 1: Introduction 


The world’s population is about 5.6 billion and is currently growing by nearly 
2 per cent a year - at least a billion more people every 10 years. The rate of 
growth in developing countries is generally four times faster than in developed 
countries and the number of people living below the level of absolute poverty is 
increasing. 


The people of developing countries are at high risk of disability and death as a 
result of disease. For example, over three million people have developed AIDS 
and some 14 million to 18 million adults are thought to be HIV-infected; about 
110 million people develop malaria every year; and over eight million new cases 
of tuberculosis, with three million deaths, occur annually. 


Developing countries’ limited budgets, coupled with economic, political and 
technical constraints, hamper progress in meeting health care needs and in 
providing family planning services. Many developing countries are currently 
struggling to finance accessible health care facilities from a budget of less than 
£7 a person a year. Most countries are now committed to taking action that 
would lead to stabilising population growth through improved reproductive 
health rather than by imposing population control measures. 


There have been three United Kingdom ministerial policy initiatives in the 
health and population sector in recent years. First, in 1987, more support was 
directed to primary health care and less to hospital building; and in 1991 and 
1994, increased support was provided to those who want children by choice. As 
a result of these policy changes the pattern of health aid has changed in the last 
decade: from support for hospital services to primary health care in the 1980s, 
to improvements in the management of health care delivery in the 1990s. 


In September 1994 delegates from 180 countries, including the UK, attended 
the International Conference on Population and Development. It adopted a 
strategy to enable all women and men, within the next 20 years, to have the 
opportunity to choose the number and timing of children and to enjoy better 
reproductive health - already embodied in the UK’s children-by-choice strategy. 
The approved programme of action envisages the need for educational, health 
care and poverty-reducing activities as well as the need for more and improved 
family planning services. The strategy calls for a four-fold increase in the 
investment in reproductive health and family planning to $21.7 billion a year by 
2015. Responsibility for implementing the recommendations rests with the 
individual countries. 


The Overseas Development Administration estimate that over 300 million 
people do not have adequate access to satisfactory health services and that over 
100 million men and women have no access to family planning services. Health 
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and population activities are therefore a priority of the aid programme and the 
Administration liaise with recipient governments to assist in the development of 
appropriate health policies. 


The Administration’s objectives for health and population activities are to 
enable those most in need to gain access to essential health care, particularly 
reproductive health care, in ways which make efficient and effective use of 
available resources. Their priorities, developed over the last four years, are to 
achieve this by focusing on: 


e supporting health sector reform; 


e encouraging children by choice by helping to provide access to, and guidance 
on the use of, better reproductive health services; 


e developing the use of cost effective technologies to reduce the numbers 
suffering from malaria, tuberculosis and HIV; and 


e helping with better health care in emergency situations (not covered by this 
report). 


UK assistance takes the form of bilateral project aid to specific countries - 
sometimes co-financed with other donors and often partly financed by the 
recipient government - and contributions to multilateral aid organisations. The 
Administration also provide support for research and give grants to UK 
non-governmental organisations funded out of the joint funding scheme. 


The Administration’s geographical desks have responsibility for bilateral 
programmes, supported by professional advisers who may also act as project 
officers. The Chief Health and Population Adviser is responsible for supervising 
the quality of the professional input provided to programmes by health and 
population advisors and other specialists, and for ensuring the adequacy of 
other systems set up to provide health and population professional inputs to 
programmes funded by the Administration. His staff administer funds for, and 
undertake value for money checks on, multilateral agencies and general 
research projects. Over the last four years the Administration have assigned 
14 professional health and population field managers to countries in which 
priority is given to health and population aid. Support for these managers is 
being developed through four resource centres which provide consultancy 
advice on all aspects of project design, implementation and impact. 


Over the past six years the Administration have spent'£911 million in the sector 
- including £393 million on emergency aid (Figure 1), allocating a growing 
proportion of the aid budget to these activities. In 1994-95 the Administration 
planned to spend about £100 million on health and population assistance, 
excluding emergency aid. 
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Figure 1: Health and population expenditure since 1988-89 
1988-89 1989-90 1990-91 1991-92 1992-93 1993-94 Total 


£m £m £m £m £m £m £m 
Bilateral 30 aa 53 58 63 72 327 
Multilateral 16 22 28 26 28 39 159 
Research 2 3 6 6 7 8 oe 
Emergency aid Zo 31 37 82 96 124 393 
TOTAL 71 107 124 172 194 243 911 
at 1993-94 prices 100 131 141 185 200 243 


During those six years the Administration provided bilateral aid in this sector to 
70 countries; some 33 per cent went to Africa, 22 per cent to Asia and 

34 per cent to multilateral health and population activities (Figure 2). To 
maximise the effectiveness of resources used, and to ensure that projects are 
more tightly managed, priority is now given to 20 countries (Figure 3). 


Figure 2: Health and population aid 1988-94 - excluding emergency aid 
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Over the last six years most health and population aid has been provided to Africa and Asia 


Figure 3: The Administration’s health and population priority countries 


Africa Ghana, Kenya, Malawi, Nigeria, Namibia, South Africa, Tanzania, 
The Gambia, Uganda, Zimbabwe, Zambia 

Asia Bangladesh, Cambodia, India, Nepal, Pakistan 

Former Soviet Union Kazakhstan, Kyrgyzstan, Russia 

Latin America Peru 
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Scope of the National Audit Office examination 


Against this background the National Audit Office sought to examine the: 
e Administration’s project monitoring; and 
e effectiveness of aid, including its impact and sustainability. 


The National Audit Office examined 13 bilateral projects and three joint funding 
scheme projects in four countries as illustrative examples: seven in Asia (India 
and Pakistan) and nine in Africa (Kenya and Zimbabwe). These countries 
received 26 per cent of the Administration’s country-specific health and 
population expenditure between 1987 and 1993. The National Audit Office also 
examined expenditure on multilateral aid and research activities. 


The 16 projects examined were selected in concert with the Administration to 
provide an appropriate cross-section. These projects included two approved in 
1994 to cover the latest procedural developments. 


The National Audit Office visited the four countries selected, meeting 
government and non-governmental organisation staff. They interviewed 
Administration staff in the regional development divisions in Nairobi and 
Lilongwe and in each of the British High Commissions. A consultant from the 
Indian Centre for Population and Development Studies provided regional 
expertise and a consultant from the Liverpool School of Tropical Medicine 
provided advice on all aspects of the study, including the case-study projects. 


The most significant changes in the Administration’s arrangements for 
designing and implementing projects in recent years have been the introduction 
of project frameworks in 1986, which formalised objective setting for each 
project; concept notes in 1992, which provide summary proposals for possible 
projects; and the “Project Cycle Management” approach in 1993 - a 
computerised project design tool, incorporating a logical framework to improve 
implementation, planning, monitoring and evaluation. The three main levels are 
illustrated in Figure 4. 

Figure 4: Responsibilities under the Project Cycle Management approach — = — 


Project structure Responsible professionals Responsibility 
Goal Chief Adviser Consistency with the Administration's 
sector policy 

Purpose* Health and To ensure that project purpose is achieved 
Population through monitoring of outputs and that outputs 
advisors lead to achievement of purpose (usually via 

Inputs mid-term review) 

Activities Field managers To ensure that project outputs are achieved 
and country through monitoring of inputs and activities. 

Outputs co-ordinators 


* Known as immediate objectives under the previous arrangements 
This illustrates the three levels in setting objectives, monitoring and impact assessment 
under the Project Cycle Management approach. 
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The Project Cycle Management approach aims to ensure that appropriate 
objectives and indicators of success are defined; measurable indicators of 
success defined; key activities identified; critical assumptions underlying the 
project specified; achievements verified; and required resources adequately 
defined. Project Cycle Management incorporates a group approach to project 
development and management and involves greater participation by project 
beneficiaries and other key stakeholders in needs assessments, planning, 
implementation and evaluation. Because of the recent introduction of this 
approach none of the projects examined by the National Audit Office used it 
though Pakistan Population HI applied the Project Cycle Management format for 
its project framework. 


The Administration make a distinction between project level activities and 
sector level assistance. The latter can consist of programme aid, aimed at 
general balance of payments support, or technical co-operation and financial 
aid directed exclusively at a sector, aimed at achieving policy changes, 
institutional development, rehabilitation and investment in the sector. This 
report covers project aid and sector aid directed towards the health and 
population sector. 


In some of the cases examined by the National Audit Office the Administration 
have adopted a process approach to projects to improve in-country participation 
and ownership. Under this approach the goal (wider objective) and purpose 
(immediate objective) are set out initially, leaving the Administration to define 
the general indicators of achievement during project development. In all 
projects the first phase activities should be fully designed, together with 
associated targets and indicators, and at each review stage the project 
framework should be updated. 


The results of this case-study examination are set out in two parts: 

e monitoring and implementation (Part 2); and 

e impact and sustainability (Part 3). 

Appendices 1 to 4 cover the case studies for the main bilateral country projects 


examined in the four countries visited. Appendix 5 deals with multilateral 
expenditure and Appendix 6 includes case summaries of research expenditure. 
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Part 2: Monitoring and 
implementation 


Health and population activities should have specific measures against which 
the achievement of immediate and wider objectives can be assessed. As there 
are many influences on people’s health, establishing objectives that a project 
can realistically achieve and which progress can be monitored against is not 
always easy, but important. To enable assumptions to be tested before a project 
is complete, and if necessary re-focus efforts and take remedial action, the 
Administration also need indicators of progress. All indicators should be 
practical, independent, targeted and objectively verifiable; and they should 
specifically define the quantity, quality and timescale for the achievement of the 
stated objectives. 


In their examination of each of the cases selected for review, the National Audit 
Office made a judgement of the extent to which the Administration had achieved 
these goals. The National Audit Office considered that all projects included 
health and population impact objectives, though in six cases these were either 
unclear or the objectives were outside the scope of the project - Contraceptive 
Social Marketing, Cervical cancer, India (Appendix 1A) Population I & II, 
Reproductive Health Services, ADB III Health, Pakistan (Appendix 2A-C), and 
Isiolo Hospital, Kenya (Appendix 3A). 


For most of the projects examined contracted managers were given the 
responsibility for managing projects and tracking progress. The 
Administration’s professional advisers undertake the majority of work relating 
to project appraisal and design. All projects were monitored by the 
Administration’s geographical desks, closely supported by in-house professional 
advisers. Frequently the Administration also employed consultants to visit 
projects on their behalf to undertake monitoring tasks. Financial management 
was undertaken by the geographical staff. 


Project management 


Management arrangements vary between regions and, in the light of past 
experience, greater capacity is being introduced with the appointment, over the 
last four years of 14 specialist health and population field managers. In addition 
increasing use is also being made of contractors to manage individual projects 
on behalf of the Administration. These arrangements will need to be evaluated 
in due course to see if they are more efficient and effective. 
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In south Asia, the Administration normally employ project managers - in India 
and Pakistan usually the British Council. In Africa, the Administration tend to 
manage projects themselves except for procurement, which they contract out. In 
Kenya and Zimbabwe they relied on staff in their regional offices in Nairobi and 
Lilongwe to visit projects and monitor progress, although since 1994 they now 
have field managers in post. 


The Administration cannot identify the comparative costs of their alternative 
management arrangements as they do not have a system to allocate staff costs 
to individual projects. An efficiency scrutiny in 1992 recommended that a time 
recording system be introduced and the Administration are reviewing the 
outcome of a pilot scheme implemented in other aid sectors in 1993. 


Progress reporting 


Progress reporting is assisted by a reporting format which identifies progress 
towards objectives and highlights key areas for attention. The source of 
progress reports varies according to the method of project management. Where 
it is contracted out or undertaken by other agencies, the Administration’s staff 
continue to visit projects regularly for monitoring purposes but rely on reports 
submitted by managers for detailed information on progress. 


The Administration expect project management contractors to report regularly 
on the extent to which objectives are being achieved or on areas requiring 
remedial action. For five of the six contracted out projects examined, the 
Administration had not agreed a reporting format with the project manager 
which would provide the necessary information. Managers’ reports often 
included considerable detail but did not provide a ready summary of progress 
and/or an analysis of expenditure against targets. 


Similarly, where the Administration’s field staff managed projects, their 
progress reports to the geographical desk did not use a common format and, in 
the National Audit Office’s view, did not cover all key areas. 


Objective and target setting 


The Administration’s framework for project design under their Project Cycle 
Management approach (paragraph 1.17) requires that objectives be set at three 
main levels: the wider goal to which the project contributes; the specific 
purpose which the project is intended to achieve; and the various outputs which 
contribute to achieving the purpose, and for which the project’s inputs and 
activities are designed. 


The Administration have not always been able to set measurable indicators of 
progress where the aim has been to improve the general health status of a 
population, particularly before the projects start. Difficulties are also 
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experienced when a “process” project’s emphasis is, for example, on 
institutional development. Despite these difficulties the Administration aim to 
ensure that realistic and measurable targets are set for proposed projects. 


The National Audit Office found that in nine of the 16 projects examined, the 
Administration had not set measurable indicators of progress towards the 
project’s objectives. In some instances the Administration had set goals to 
improve health status, which the National Audit Office consider were too broad 
to demonstrate project impact and too difficult to monitor. Some of the 
indicators selected were designed to enable impact to be assessed after project 
completion. In other cases, the indicators measured project achievement at 
output level - for example the number of clinics built - but not at impact level - 
such as clinic usage and treatment provided. Such indicators do not enable the 
Administration to test their assumption that planned outputs will lead to the 
achievement of project objectives. 


The National Audit Office noted, however, an improvement over time. In 
particular, in the two recently approved projects (Appendices 2D and 4C) the 
Administration had designed more realistic and quantifiable indicators with 
data that are more readily available and useful to monitor progress towards the 
stated objectives. Whilst the Zimbabwe Sexual Health project (Appendix 4C) set 
appropriate objectives and useful indicators, some of the targets appear 
ambitious. They rely on critical assumptions about condom use that were not 
well documented. The project assumes that the supply and distribution of 
condoms will result in the usage rate increasing to 25 per cent which in turn 
will be sufficient to hold HIV infection at its current level. Factors external to the 
project may result in an increase in HIV infection despite achieving the planned 
increase in the use of condoms. 


In 1993 the Administration initiated a major programme of training in the 
procedures for the regular review of project objectives and the resetting of 
indicators during implementation. These revised procedures are now being 
introduced by health and population advisers. 


Financial management 


Although financial monitoring is the responsibility of the geographical desks, in 
most instances they did not have a sufficiently detailed knowledge of a project 
on which to exercise financial control and need to rely on reports submitted by 
project managers. In most cases financial records were up-to-date but they did 
not always include details of individual project components. For example, the 
Administration did not have a breakdown of expenditure by component for 
Pakistan Population I and II (Appendix 2A) or, until the fourth and final year, for 
St Stephen’s Hospital, India (Appendix 1B). 


In more recent projects, such as health management strengthening and 
Population III (Appendix 4A and 2D), delays had also been experienced in 
obtaining expenditure analyses by component. In all these cases the 
Administfation did not have ready access to the necessary information to 
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exercise judgements about efficiency and effectiveness. For St Stephen’s, they 
had to delay a decision on whether funds should be made available to pay for 
additional training. 


The introduction of overseas field managers should provide the opportunity to 
improve the financial control of projects, provided the managers are given clear 
terms of reference defining their financial responsibilities and access to the 
Administration’s computerised financial information. 


Contractual arrangements 


In nine of the 13 projects examined where management, monitoring or 
procurement tasks had been contracted out, the Administration had not sought 
competitive tenders. Two of these nine were relatively small - under £50,000 
each - a level below which the Administration may waive tender action if the 
cost of holding a competition is considered significant. The other seven 
contracts ranged from £100,000 to £1.5 million; competition was waived 
because the project was an extension of a previous phase, the manager had 
unique expertise in the area, or because the Administration aimed to give the 
British Council the opportunity of first refusal or because their contractual 
arrangements gave the Crown Agents exclusive rights to Administration - 
funded procurement for certain countries. 


When seeking project management tenders, the Administration ask contractors 
to submit proposals on the basis of a project outline and brief terms of 
reference. They then negotiate any adjustments they consider necessary in the 
fees or services. While this gives contractors who are familiar with the 
procedures an advantage in submitting proposals, it removes the 
Administration’s need to define in detail the management services they require. 
The Administration consider that this flexibility can, particularly in process 
projects, be an advantage. 


Although the Administration had a broad Memorandum of Understanding with 
the British Council covering project work carried out on the Administration’s 
behalf, in two cases adequate contract arrangements with the Council had not 
been made. In Orissa Health and Family Welfare (Appendix 1A) the 
Administration obtained a management proposal from the British Council but 
the service contract was not signed until two and a half years into the project - 
by which time the Council had been paid more than £0.7 million. In the Cervical 
Cancer Research project (Appendix 1C), the Administration employed the 
British Council as project manager at a cost of £37,000 but no terms of 
reference were agreed or contract signed. 


Adequate contract arrangements were in place for procurement projects but 
two had problems: 


Emergency Supply of Condoms, Zimbabwe. The Crown Agents were 
appointed without competition, despite having more limited experience in 
procuring such contraceptives. Although the first three consignments passed 
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pre-delivery inspection, the Zimbabwe regional drug control laboratory 
found the condoms to be defective. The Administration have paid the 
suppliers, Dongkuk Industries, £0.4 million for two of the consignments. 
Dongkuk have agreed to replace the defective consignments free of charge 
over one year ending November 1995 and the contract with Crown Agents 
for this project has been terminated (Appendix 4B). 


Supply of Vehicles to Pakistan (Population I and II). The Administration 
and the Pakistan Government agreed with a commercial supplier what 
emerged to be an inadequate specification for 91 vans procured at a cost of 
£885,000. These vans needed substantial modification, at an additional cost 
of £22,500. As a direct result of this modification about 75 vans suffered 
significant damage which cost the Administration a further £16,700 for 
repairs (Appendix 2A). 


In these cases Administration staff spent considerable amounts of time 
discussing possible settlements internally and with the procurement agent 
regarding the negotiation of settlements with suppliers. 


Monitoring 


Effective project monitoring aimed at identifying and addressing 
implementation problems, should be established at the design stage. Monitoring 
criteria set at this point should include: ensuring sufficient information is 
provided to track project implementation; setting quantitative and verifiable 
indicators of project progress and performance; establishing responsibility and 
providing funding for monitoring; and, where it is essential to identify actual 
performance, establishing of baseline data. 


The National Audit Office found that satisfactory monitoring criteria had been 
set for 13 of the 16 projects. Three projects had had formal mid-term reviews, 
two of which were carried out jointly with the World Bank. The cost of 
monitoring was not, however, identifiable as the Administration do not record 
staff time allocations. The Administration have not provided guidance on project 
monitoring requirements, including how to relate a proposed monitoring 
strategy to the project’s cost and perceived risks. This is being addressed in the 
Administration’s current revision of their Guide to Aid Procedures. 


The National Audit Office considered that project monitoring tended to focus too 
much on the provision and use of inputs and often did not relate progress to 
objectives and targets. This was mainly because of the need to address 
problems with the delivery of the required inputs (mainly timeliness and 
quality) and monitoring had to concentrate at that level. Where mid-term 
reviews were undertaken, however as in the case of Orissa (Appendix 1A), there 
was greater attention to outcomes in relation to the objectives and targets set. 
The Administration indicated that mid-term reviews were designed to focus 
particularly on whether project outputs were achieving the purpose as specified 
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in the project framework, and that routine monitoring missions should be 
primarily concerned with whether project activities and inputs were achieving 
the specified outputs. 


Remedial action 


When projects are not meeting their planned targets, remedial action needs to 
be considered. For projects where the management was contracted out, the 
Administration intervened only at higher levels in recipient institutions, or when 
delays were particularly serious. However they did not specifically define 
respective responsibilities for remedial action in their contracts with project 
managers. In contrast, where the Administration’s field staff managed projects, 
they had responsibility for remedial action and informed the geographical desks 
of any serious problems. 


The National Audit Office noted: 


a) four cases where remedial action had been successful, resulting in prompt 
improvements in progress: 


Family Planning through community institutions, Pakistan, where the 
Administration noted in 1992 that 97 per cent of sterilisations under the 
project were for women. They encouraged the Family Planning Association 
to focus more on men and by September 1993 the proportion of male 
sterilisations had risen to 40 per cent; 


St Stephen’s Hospital, India, where persistent pressure by the British 
Council’s building adviser should ensure that the new wing will be completed 
only eight months late - substantially earlier than had appeared likely 
(Appendix 1B); 


Isiolo Hospital, Kenya, where the additional technical assistance provided 
after the construction enabled a hospital preventative maintenance system to 
be introduced and a surgical referral service established (Appendix 3A); and 


Kenya Population III (family planning association component), where the 
provision of an independent architect reduced the construction costs by 


£65,000 (Appendix 3B). 


b 


— 


in most other cases the Administration or project managers took remedial 
action which was partially successful or was eventually effective - for 
example: 


Orissa Health and Family Welfare, India, where, during two years of 
minimal progress, the Administration and the British Council repeatedly 
requested the recipient authorities to comply with'the project conditions. 
Faster progress was noticeable after a further 18 months due to a significant 
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increase in the Administration’s monitoring. Despite these delays the 
Administration nevertheless expected the project to meet its original 
construction target of March 1995 by June 1995 (Appendix 1A); 


Asian Development Bank III, Pakistan, where the British Council have 
repeatedly pressed a provincial government to authorise staff posts for repair 
and maintenance workshops to be set up. In May 1994 the Administration 
formally suspended their inputs under this component and have since 
received assurances that the posts will be established (Appendix 2C); 


Zimbabwe Health Management Strengthening, where there were delays of 
a year and two and a half years in appointing two technical advisers. In the 
latter case the delay was in part caused by the Administration’s desire not to 
undermine the project by recruiting an unsuitable person and partly because 
they had initially set over- ambitious terms of reference for the post. With 
these problems behind them Administration consider that the project is now 
performing well and expect it to make a major contribution to health 
management development in Zimbabwe (Appendix 4A); 


Kenya Population III, where the project co-ordinator did not perform 
satisfactorily. After three years seeking to persuade the World Bank to amend 
his work objectives so that they provided greater critical dialogue with 
Kenyan institutions, the Administration withdrew funding for this post. The 
Administration consider that the co-ordinator’s performance neither assisted 
nor adversely affected the project and that its main components are now 
providing tangible results which should lead to a marked improvement in 
reproductive health services (Appendix 3B); and 


Emergency Supply of Condoms, Zimbabwe, where the Administration 
appointed an alternative procurement agent to supply condoms after the first 
consignments were found to be defective. The replacement condoms began 
to arrive 18 months after the planned supply date (Appendix 4B). 


three cases where remedial action was not taken or was taken late because 
of over-optimistic objectives, professional dispute, and a decision by the 
Administration to delay action. 


Pakistan Population I & II, where a number of the components made 
limited progress: for example, the innovative and non-governmental 
components, where only a small percentage of their respective £515,000 and 
£600,000 allocations were spent. This was partly because of political turmoil 
in the country, the Administration having no field presence for over three 
years and no remedial action having been taken to address the delays 
caused by local bureaucracy and its insufficient implementation capacity 
(Appendix 2A); 


Cervical Cancer Research, India, where the Administration had not 
formally allocated responsibility for remedial action and disagreements 
between consultant specialists about the scientific approach hindered 
progress (Appendix 1(); 
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St Stephen’s Hospital, India, where the Administration took no action on 
the planned financial consultancy until March 1994, when the project had 
entered its final year. Steps are now being taken to strengthen the hospital’s 
financial management but the Administration have still not established 
whether the hospital has the capacity to sustain services to the poor, as 
expected in a 1990 study (Appendix 1B). 


The delays identified in these cases reflect, in part, the difficulty in 
implementing prompt remedial action in partnership projects. In these 
circumstances the Administration have, on occasions, decided to delay remedial 
action and instead to review the objectives and activities supported. 


Contributions to multilateral organisations 


The Administration contribute on a voluntary basis to the health and population 
activities of three international agencies: the United Nations Population Fund, 
the International Planned Parenthood Federation and the World Health 
Organisation. Contributions to the World Health Organisation include funding 
for health research activities (Part 3). 


Since April 1988 the Administration have contributed some £159 million to 
these organisations. The National Audit Office examined how the 
Administration monitor the value for money they obtain from this funding. They 
concluded that the Administration: 


e had established comprehensive review and evaluation procedures, providing 
management with valuable information on the effectiveness of the funded 
agencies; 


e consider a broad span of evidence, including evaluations performed by other 
donor countries, in assessing the effectiveness of funded agencies; and 


e had been active in seeking the reforms necessary to ensure that their 


increasing contributions to World Health Organisation’s special programmes 
achieve the optimum value for money. 
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Part 3: Impact and 
sustainability 


In assessing projects, the National Audit Office recognised that many operate in 
difficult conditions often in deprived areas with weak and ineffective 
management systems - while attempting to tackle major and urgent health and 
population problems. Projects are also often implemented in difficult political 
and economic environments. Whilst individuals may benefit when projects 
improve services, the effects may not always be directly quantifiable through 
general measures of health status. 


Impact 


Project impact arises from the effects - direct and indirect, wanted and 
unwanted - which result from a project’s activities. Comprehensive analysis of 
the impact, or potential impact, of health and population projects is difficult but 
evaluation studies can measure the achievement of specific targets, collect 
qualitative information from implementers and beneficiaries, and draw on 
national and local surveys producing health and population indicators. Research — 
can contribute to effective monitoring by testing whether indicators are 
reasonable proxies of impact. Impact becomes complex and costly to assess if 
attempts are made to disaggregate the Administration’s contribution to the 
overall impact, such as in partnership projects. 


Impact, or potential impact, can be assessed at a number of stages and ina 
number of ways: at the design stage (paragraph 3.5), in progress reports 
(paragraphs 3.6 to 3.8), by comparison with objectives (paragraph 3.9), in 
completion reports (paragraph 3.10), through general research (paragraph 
3.11) and in post completion evaluation reports (paragraph 3.12). During 
implementation many factors can influence project impact (paragraphs 3.13 
and 3.14). 


In some of the projects examined the Administration had either delayed 
carrying out an impact assessment - for example, Isiolo Hospital (Appendix 3A) 
- or had not carried out any impact assessment during the project’s life - such as 
Kenya Population III (Appendix 3B). The Administration said that the latter 
project would be included in post-project impact assessment studies currently 
planned. In seven other cases, such as the non-governmental organisation 
components of Pakistan Population I and II, the Administration did not ensure 
that sufficient information was provided to assess impact. 
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(a) Design stage 


At the design stage the Administration aim to make a general assessment of the 
likely impact of the proposed projects, where appropriate drawing on the 
impacts achieved by earlier phases or similar projects. Only two of the projects 
examined included explicit consideration of whether the interventions proposed 
were the most cost-efficient. Orissa (Appendix 1A), for example, is based on the 
Government of India’s policy assumption that building static health centres, 
together with training and other inputs, is the most cost-efficient means of 
improving health services in rural areas. The Administration did not undertake 
a rigorous examination of alternatives before commencing Phase II as they 
perceived that mobile clinics (where an earlier project had failed) were not a 
viable option to static services and considered that no other choice existed. 
They also saw the project as an opportunity to bring Orissa up to 50 per cent of 
the Indian Government’s recommended national provision for static services. 


(b) Progress reports 


As identified in paragraph 2.25, the Administration’s routine monitoring reports 
focus on the provision of inputs and progress in achieving project outputs. In 
most cases, they do not include analyses of the impacts - positive and negative - 
that a project is achieving, or likely to achieve. Such issues should be examined 
during mid-term reviews. In addition, some separately commissioned 
evaluation reports on particular on-going projects or project components 
contained useful analyses of impact: the reports on the vehicles (1988) and 
contraceptive supply (1990) components of Pakistan Population I and II 
(Appendix 2A), and the Kenya Family Planning Services project (1992) 
(Appendix 3C) are examples. 


The Administration have not carried out many impact assessments of the 
training components of projects. For example, there has been no assessments of 
the training aspects of Pakistan Reproductive Health Services (Appendix 2B), 
Kenya Population III (Appendix 3B) or Zimbabwe Health Management 
Strengthening (Appendix 4A). 


Most of the training provided was UK-based, even though this is more 
expensive than in-country or regional training. Much of it has been of a high 
standard but recipients have not always been able to use their new skills to the 
full. With Orissa (Appendix 1A) and Asian Development Bank III (Appendix 2C), 
some of the staff who received training in the UK were, soon after their return, 
transferred to jobs where they were not able to use their new skills. 


(c) Achievement of objectives 


The success of a project is generally determined by whether it is achieving, or 
has achieved, its objectives. To make this assessment it is necessary for clear, 
appropriate, project-specific objectives be set at the design stage and for data to 
be collected during and after the project. In the 16 projects examined the 
National Audit Office found: 
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e four cases where separate objectives for the scope of the Administration’s 
funded project had not been set. This included two projects (Pakistan 
Population I and II and the Asian Development Bank III) where the 
Administration’s contributions were linked to multilateral projects, each of 
which had its own detailed objectives; and 


e seven projects lacked the necessary baseline data. For six of the indicators 
for which targets were set in the Zimbabwe Sexual Health project 
(Appendix 4C) there were no reliable data on indicator levels at the outset of 
the project. Studies planned by the Administration to produce baseline data 
for the Zimbabwe Health Management Strengthening project were delayed 
by three years. 


(d) Completion reports 


The Administration require completion reports on projects costing over 
£250,000, primarily to identify successful and unsuccessful elements of 
implementation, assess impacts achieved or expected and to derive lessons. 
However, reports are often not produced until long after a project has finished, 
limiting their usefulness. At the time of the National Audit Office examination, of 
seven completed projects only one report had been produced; three were in 
draft. 


(e) General research 


Various studies have been undertaken on the Administration’s behalf to assist in 
predicting the potential impact of different forms of health and population 
interventions. Recent work has included assessing the best public/private mix 
for health care in developing countries, comparing different types of cost 
recovery systems, and reviews of the effectiveness of health service 
interventions. The Administration have recently decided to put more effort into 
summarising the findings of this kind of research so that it can be more easily 
disseminated. 


(f) Evaluation reports 


The Administration’s Evaluation Department undertake ex-post reviews of a 
number of projects each year. In the last five years they have evaluated two 
small health projects. In one, on water supply and sanitation, it proved 
impossible to quantify the project’s health impacts because of inadequate 
objectives and a lack of baseline data; in the other, on children’s nutrition units, 
they concluded that the project had had a positive impact on its intended 
beneficiaries, although they questioned the project’s sustainability. The 
Administration plan to evaluate four population projects over the next two years. 


(g) Factors affecting impact 


The benefits derived from a project should be assessed against the cost involved 
to establish the extent to which value for money has been achieved. The 
Administration do not charge design and appraisal cost to projects or the time 
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of their staff. In addition, for four projects, the Administration did not readily 
know the amount spent on individual components; one of these (Pakistan 
Population I and II) pre-dated the Administration’s management information 
system. 


3.14 A number of factors can affect implementation and project impact: adequate 


aS) 


3.16 


funding arrangements, the timely appointment of technical staff, the timely 
delivery of project equipment and recipients’ commitment to the project. 
Examples of where problems arose include: 


e poor project design leading to reduced project impact because the activities 
lacked coherence, and did not focus on one clear purpose (Pakistan 
Population I and II, Appendix 2A); 


e lack of commitment by the implementing agency in Orissa (Appendix 1A) 
caused delays in the new buildings becoming operational; 


e failure to use inputs effectively - a minibus in Kenya (Appendix 3B) has 
remained unused for three years, and a Landrover supplied under the Health 
Strengthening Project in Zimbabwe (Appendix 4A) remained unused for two 
years until the technical officer arrived; and 


e procurement problems led to the failure to deliver 66 million condoms to 
Zimbabwe (Appendix 4B) and resulted in stocks falling to a critically low level 
which a follow-on project is beginning to meet. 


(h) Findings 

Of the 14 projects that had started, the National Audit Office found that nine had 
largely achieved, or were likely largely to achieve, their objectives, three will 
partly achieve their objectives and two were likely to have little impact partly. 
The Kenya Family Planning project (Appendix 3C), for example, had been a 
major success in increasing the use of injectable contraceptives; the Kenya 
Isiolo Hospital (Appendix 3A) had provided a much needed facility; Orissa Phase 
II (Appendix 1A) has helped to establish the infrastructure for future health 
services for over 30 million people; St Stephen’s Hospital (Appendix 1B) is likely 
to provide greatly improved services; and the Zimbabwe Supplementary Supply 
of Condoms (Appendix 4B) is now ensuring access to condoms. On the other 
hand, the initial Zimbabwe Condom Supply project (Appendix 4B) and Pakistan 
Population I and II (Appendix 2A) made little progress. 


Sustainability 


Sustainability is the capacity to maintain the desired impacts beyond the end of 
the project. The Administration have well-established procedures to appraise 
the economic viability of projects but most proposals appeared to pay 
insufficient attention to how activities could be sustained after UK funding came 
to an end. The Administration told the National Audit Office that detailed 
discussions about sustainability always took place internally and with recipient 
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countries. The issue was also often addressed through donor agencies’ wider 
dialogue with national agencies but much of this is not documented on project 
files. The Administration are currently preparing a policy statement about 
health and population sustainability. 


The Administration recognise that projects should always consider 
sustainability. They however, accept that many developing countries are unlikely 
to be able to meet the full cost of their contraceptives, essential drugs and 
medical supplies. They therefore plan this kind of project on the assumption 
that donors will continue to contribute to such supplies. Where the aim of a 
project is to achieve a high immediate impact there may be less need to 
consider sustainability - for example a project addressing the spread of HIV. 


Even where the Administration gave specific project consideration to 
sustainability, recipient governments were, in many instances, reluctant to 
commit themselves fully to the continuing cost of providing medical and 
contraceptive supplies. Given the decreasing levels of funds available for health 
care in developing countries, the Administration consider that this situation 
could prevail for several decades. 


Examples where project sustainability was not considered include the Pakistan 
Reproductive Health Services project (Appendix 2B) and the Population I and II 
non-governmental schemes (Appendix 2A). In the latter case, 35 of the 

171 non-governmental organisation outlets funded closed down after the project 
finished. 


Three of the six contraceptive supply projects examined provide examples of 
different levels of sustainability consideration at the outset: 


e in Pakistan Population III (Appendix 2D) sustainability was considered and 
action taken. The Government of Pakistan is committed to contributing a 
specified and increasing proportion of the national population budget over 
the life of the project. The Administration will not release project funds . 
unless these commitments are honoured; 


e in Zimbabwe Sexual Health (Appendix 4C) sustainability was considered 
but the risks remain high. The Government of Zimbabwe has expressed an 
intention to pay for an increasing percentage of condom procurements but 
the Administration are not in a position to produce safeguards to ensure such 
contributions are paid; and 


e in Kenya Family Planning Services (Appendix 3C) sustainability was not 
considered. Kenya is largely reliant on donors for contraceptive supplies and 
the Administration have given thought to ways to reduce this long-term 
dependence without requiring Kenya to borrow substantial sums, including 
gradually introducing some user charges and expanding contraception 
distribution. 
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The National Audit Office consider that only three of the 13 projects examined, 
where activity has started, have good sustainable prospects in the sense that 
they will lead to continued services and benefits without external financing - 
Pakistan Family Planning, St Stephen’s Hospital (Appendix 1B) and Zimbabwe 
Health Management Strengthening (Appendix 4A). 


Three main factors can have a significant influence on sustainability: 


e the degree of involvement and ownership by the recipient government or 
institution and beneficiaries (paragraphs 3.23 to 3.25); 


e the management capacity of the recipient government or institution 
(paragraphs 3.26 and 3.27); and 


e the availability of recurrent funds (paragraphs 3.28 to 3.32). 


(a) Involvement and ownership 


The Pakistan Family Planning project is a good example of how working closely 
with the beneficiaries from the outset increases the prospects of sustainability. 
As well as providing land for the new community centres, the local people have 
accepted responsibility for the buildings’ on-going maintenance. Similar 
commitments have been made in the case of the refurbished clinics being built 
under Kenya Population II (Appendix 3B). 


In most of the projects examined the National Audit Office found a high level of 
ownership on the part of recipient institutions. Examples include the 
commitment by the Zimbabwean Ministry of Health and Child Welfare for the 
Health Management Strengthening project (Appendix 4A), despite delays in 
obtaining the planned support from the Administration; and the Family 
Planning Association of Pakistan’s commitment to the community institutions’ 
family planning project. 


There is no evidence that the Administration attempted to secure the 
commitment of the Ministry of Education to the population studies centres 
which formed part of Pakistan Population I and II (Appendix 2A). The Ministry 
have not taken over their funding and one of the two centres closed down in 
1993; the future of the other is uncertain. There was also a lack of ownership in 
Asian Development Bank III (Appendix 2C) by the Government of North West 
Frontier Province, Pakistan, which suggests that some project activities may not 
be sustainable. 


(b) Management capacity 


Since 1991, improving the management capacity of recipient country health 
institutions has become a priority area but the Administration’s efforts have not 
always been successful. For example, attempts to strengthen the Kenyan 
Population Council (Appendix 4A) have achieved little. A more flexible approach 
to planning - the process approach - is being used to develop management 
capacity. As this approach becomes better developed it will need evaluating. 
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Since the inception of the Administration’s project, the effectiveness of the 
Administration’s inputs to Orissa’s health sector (Appendix 1A) has been 
undermined by limitations in local management ability. The Administration 
were aware of this problem from the earlier phase and the mid-term review of 
phase II in January 1993 identified a need to strengthen management. The 
current project now includes pilot work on central and district health 
management information and training systems, the results of which should 
inform the design of a successor project. 


(c) Recurrent expenditure 


The Administration are reluctant to continue funding recurrent costs because of 
the long-term commitment which this would involve and their desire to 
encourage developing countries to help themselves. Equally, recipient 
governments are often reluctant to commit themselves fully to the continuing 
costs. 


Most project proposals did not include estimates of the recurrent cost 
implications and the potential future burden on the recipient’s budget. However, 
five projects specifically considered recurrent costs - for example Zimbabwe 
Sexual Health (Appendix 4C) where the Administration estimated that an 
additional £2.8 million a year would be required to sustain activities. 


For seven projects recipients have found it difficult to meet recurrent costs. The 
initial medical supplies for the Isiolo hospital (Appendix 3A) were therefore 
funded by the Administration to ensure immediate sustainability following 
construction. In Orissa (Appendix 1A) the state government is not able to 
provide adequate funds for the drugs and consumables required by the new 
centres due to its worsening financial position. Under Asian Development Bank 
III (Appendix 2C) the Administration and the British Council have lobbied for 
the project’s recurrent costs to be covered by local budgets - but so far with 
limited success. 


It is likely that most project activities, particularly those targeted at the poor, 
will not be able to generate enough income to cover all their costs. To be 
sustainable they are likely to need continued external funds from donors, the 
government or a non-governmental organisation. In contrast St Stephen’s 
Hospital (Appendix 1B) is likely to be sustainable as it is managed by an 
organisation which has proved itself to be very successful at securing funds. 
Similarly, with the backing of the Family Planning Association of Pakistan, the 
sustainability prospects of the family planning through community institutions 
project look good. . 


Although the Administration have well-established procedures to appraise the 
sustainability of their projects, it remains hard to predict for projects addressing 
institutional reforms. The Administration recognise the difficulties, which have 
recently been re-emphasised following the results of an evaluation of 
institutional development projects. They plan to provide further guidance in this 
area. 
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Impact of health and population research 


The Administration support a wide range of health and population research and 
development activities through partnerships with academic groups, 
non-governmental organisations, international bodies and the British Medical 
Research Council. Since April 1989 they have spent £85 million on health and 
population research, including £55 million on contributions to seven 
international bodies and £30 million to UK organisations. 


The main objectives of this research are to provide the information to assist in 
the establishment of a social and economic climate for people to choose when to 
have children; and to improve the availability, accessibility and quality of family 
planning and health care services. 


During their visit to India, the National Audit Office reviewed one country-based 
research project funded from the bilateral aid programme (Cervical Cancer, 
Appendix 1C). They concluded that the research project had been poorly 
designed, had been set an unrealistic objective and had been inadequately 
implemented. It had not influenced cancer policy or control in India, although it 
has helped transfer, from the UK, DNA hybridisation techniques in research. 
The Administration consider that there has nevertheless been excellent value 
for money from the £300,000 invested as it has improved screening procedures 
and the viability of the laboratory. 


The National Audit Office also reviewed the general health and population 
research projects to see what steps the Administration had taken to ensure that 
they were focused on Departmental priorities, whether they had monitored 
progress and results, whether completed research had had an impact on policy 
formulation, and whether the Administration had encouraged follow-up action, 
including the introduction of cost-effective health measures (Appendix 6). 


The National Audit Office findings were: 


e the Administration had taken significant steps to provide greater direction to 
the research currently undertaken on their behalf. They had formalised the 
arrangements with the Medical Research Council and had introduced 
competition into the selection process; 


e there had been an impact on policy formulation at country and international 
level and recently completed research had enabled research skills to be built 
up in the essential health and population areas; 


e the Administration are putting more effort into summarising and 
disseminating the findings of research studies, assessing the potential impact 
of different forms of health and population interventions; and 


e the Administration are now making arrangements for evaluating most of 
their research activities. 
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Appendix 1: India 


1 India has a population of over 900 million, growing at a rate of 2 per cent a 


year. It is expected to rise to over 1 billion by 2000. India also has almost 

40 per cent of the world’s poor. Despite faster improvements in health than in 
most south Asian countries, morbidity levels remain high among infants, 
children and women. Life expectancy is 59 years. 


India’s national health policy aims to provide health for all and to achieve a net 
reproduction rate of one by 2016. Previous initiatives have emphasised female 
sterilisation but this has been replaced by a greater focus on spacing methods 
and increased family planning and reproductive health service provision. The 
Administration have provided £22 million in bilateral health and population aid 
since 1988-89. Their current priorities are health sector reform and 
management strengthening, reproductive health, and efforts to stem 
tuberculosis, malaria and HIV. 
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A. Orissa health and family welfare, phase Il 


Wider Project Objective | - Improve the health and family welfare status of people in Orissa (population 
30m), giving particular priority to mothers and children, and tribal groups 
Increase the availability of health and family welfare services in 10 districts, 
and improve their quality and usage in all 13 districts 

Identify and test new and more cost-effective methods of health care delivery 
Increase community participation in the planning and operation of health 
services 


Immediate Objectives 


Project Activities 
(1990-95) 


1,484 buildings, including health centres, staff quarters, district training units 
In-service training for health workers 

Promotion of information, education and communication activities 
Developing a health management information and evaluation system 

Repair of buildings erected in Phase | and maintenance of all buildings 
Vehicles for project management, training and health centres 

Research and innovative projects for malaria, safe motherhood, new 
contraceptives 

Technical co-operation (UK training, consultants, surveys, management and 
monitoring costs) 


Project Cost Total cost: £26m (£6m from Indian & Orissan Governments) 
ODA Allocation: £20m (£17.2m local costs, £2.8m technical co-operation) 
Expenditure by 
November 1994: £7./7m (£5.7m local costs, £2.0m technical co-operation) 
— the Rupee has consistently depreciated against 
sterling during the project. 


Project Management Carried out by the British Council 
Management charges: £1.5m (8% of allocation, 11% of expected expenditure) 


Project history: Phase, The Administration contributed £13.3m to Phase |. 68% of expenditure was 
1980-87 for constructing or upgrading 1,400 health centres and other buildings. These 
were generally of a low standard and poorly maintained. 
The number of nurses increased three-fold; training standards were raised. 
A survey in 1989 found some improvement in service delivery since 1981 but 
people, especially the most disadvantaged, remained unaware of health 
services or did not feel the need to use them. In 1989 Orissa’s infant mortality 
was 133 per thousand compared with the Indian average of 96. 
Targets 
Achievement indicators set by the Administration did not enable them to fully monitor progress towards 
their immediate objectives or to take all necessary remedial action because: 
e many could not be quantified until after the end of the project - for example, increase in usage of 
health services was to be measured over 10 years using baseline and endline surveys; 
e — some, for example for construction and training, were quantifiable during the project and provided an 
indication of increased availability of services but did not address quality; and 
e the project framework rested on the assumption that the inputs would improve the availability, quality 
and client usage of the increased services provided. 
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A. Orissa health and family welfare, phase II 


Targets (contd.) 

After the mid-term review in January 1993 the Administration concluded that the project required a more 
precise definition of objectives and related inputs, focusing on health systems development. However 
because of the project's subsequent slow progress and a change in key personnel, they decided in 
mid-1993 to pursue only certain aspects of their new strategy and to concentrate their efforts on achieving 
the original planned outputs. The Administration nevertheless recognised that a clearer set of indicators 
was still required. 


Management and Reporting 

e In June 1989, at the Administration’s request, the British Council submitted a draft proposal for field 
management; the Administration did not seek other offers. The Administration began payments from 
January 1990 but the contract was not signed until October 1992, then covering the period April 1992 
to March 1994. 

e The contract did not provide a specific reporting format for the Council to follow. Its reports contained 
considerable detail but did not provide a readily accessible summary of progress, show expenditure 
against targets for each component, or indicate which key areas required remedial action. After 1991 
the Government of Orissa would not provide the British Council with expenditure details for each 
component. In September 1993 the Administration asked for more information on progress against 
targets but still provided no specific guidance on format. From January 1994 the-Council provided 
additional monthly update reports. 


Monitoring 

e The management contract did not provide a clear definition of responsibilities. 

e The Administration were concerned from the outset that the Orissan Government might have difficulty 
in managing such a complex project. They therefore liaised directly with the construction agency. 
From 1993 they increased the frequency of their monitoring missions from once or twice a year to 
four visits. This reflected growing problems faced by the project - in particular the building 
programme had fallen significantly behind schedule because the Orissan Government wanted to use a 
different building contractor. The project director refused to assign building sites to the authorised 
contractor until July 1993, as a result of which the building programme was Set back a year. 

e The Orissan Government was also slow to release funds to the construction agency. Administration 
pressure was initially unsuccessful and early in 1994 they considered exploring alternative means of 
funding. However the flow of money improved in mid-1994 and it was expected that the construction 
target would be met by June 1995. 

e Expenditure is expected to be £15.6 million by the end of the project (78% of allocation). 


Project progress: construction 

e About 70 % (£12.5m) of the project's local cost is intended for construction. 

e The Administration did not assess whether this mix of construction and other inputs was the most 
cost-effective means of delivering primary health care, although construction had also been a major 
component in Phase |. At the Phase || appraisal stage they considered allocating less to construction 
but concluded that it was justified by the inadequacy of buildings relative to Indian Government 
norms. However they recognised the risk that the uptake of health services might remain low, as it 
had during Phase |. They sought to address this risk by training health workers and through social 
mobilisation activities but, after the 1993 mid-term review, concluded that uptake of services 
remained unacceptably low. 

e A National Family Health Survey in 1993 demonstrated a low uptake of services in clinics in rural 
Orissa where 90% of the population lives: 

- only 5% of births were attended by paramedical health staff, many of which were not in a clinic; and 
- 85%yof rural deliveries were in homes, attended by relatives or traditional birth attendants. 
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A. Orissa health and family welfare, phase II 


Project progress: construction (contd.) 


Better sub-centre clinics were required in Phase || but the Administration still intended them to be low 
cost and simple. The buildings are of a higher specification to minimise maintenance requirements but 
are also 50% larger to allow health workers to live on the premises and to provide greater privacy for 
the medical staff to work with patients. In November 1993 the British Council buildings adviser 
proposed a simpler and 30% cheaper design but the Administration did not adopt this alternative, 
deciding not to delay construction further with changes. They did, however, test the alternative design 
and the results of this will inform a possible phase III. 

By July 1994 611 of the planned 1,484 buildings had been completed, of which only 279 were in use 
because the project authority had not released them to the health department. A further 562 are in 
progress 


Project progress: training 


Since 1989 about 60 people have been sent to the UK for training but in 1993 the project authority 
nominated students for courses without consulting the British Council. As a result, 12 have been 
rejected by the Indian Government, causing the training opportunities to be lost, and some unsuitable 
candidates have been sent. In other cases staff have been transferred to non-relevant duties on their 
return. The situation has improved in 1994 with the British Council becoming more involved although 
again some unsuitable candidates have been nominated. 

Better progress has been made on the local training component. 13 district training units have been 
equipped and district training teams have been formed. Substantial progress has been made towards 
reaching targets for training traditional birth attendants, multi-purpose health workers and other health 
Staff. 

While over 5,000 birth attendants have been trained under this project, the Administration’s target of 
raising the proportion of births attended by trained personnel from 20% to 60% over 10 years seems 
unachievable. 

The new training system was developed taking account of local needs and training material was locally 
produced. 


Project progress: other components 


Some project components have made little progress: 

- Ahealth management information and evaluation system was planned but the Administration have 
experienced long delays in agreeing with the Orissan Government the design of a family health card 
which was to form the basis of the system. An agreed card was finally released in mid-1994. 

- The Administration hoped to identify innovative projects to test new methods of health care in 
collaboration with non-governmental organisations but to date only one - a study on rice-based oral 
rehydration - has been completed. A second - impregnated mosquito nets - began in July 1994. 
Until mid-1994 the project authority did not fund any of the non-governmental organisations, 
despite pressure from the Administration; 17 organisations have since been provided with 50% of 
their funds. 

- Implementation of a component to strengthen district level health management, designed 
progressively from December 1993, started in June 1994 when initial workshops were held. 


Impact 


The Administration are concerned that the slow progress will undermine the project’s impact. Their 
and the British Council’s efforts have resulted in greater progress during 1994 and a third phase is 
being considered. 
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A. Orissa health and family welfare, phase II 


Impact (contd.) 
e —_|In August 1994 two impact studies were completed: 

- One examined the construction component. It concluded that the sub-centre based approach would 
have a considerable impact on the delivery of community health care services and found that the 
new buildings were more spacious and hygienic. The phase !| sub-centres have increased the 
take-up of family planning services and clinic-based deliveries. However the phase II centres are 
often on the village outskirts, affecting their usage and the security of the health workers; 

- the other reviewed the training component and found the general training of multi-purpose health 
workers to be too concentrated, affecting both the quality of training and the regular sub-centre 
work. The intensive training received by 500 female health workers was found to be well conceived 
and conducted. 


Sustainability 
e The sustainability of the project is in doubt, because of: 

- weak management in the Orissan health department; 

- the poor record on maintenance of health related buildings. The Administration designated £1.1m to 
fund repairs but by July 1994 only £0.8m had been spent. The average requirement for maintenance 
is Rs9 per sq ft but the Government has allocated only Rs0.8 (2 pence); 

- the under-provision for other non-salary recurrent costs, especially drugs and consumables. In 1993 
the Administration examined district budgeting for health and family welfare and found that over 
80% was allocated to salaries, leaving only Rs3 (6 pence) a head for drugs, maintenance and other 
costs; and 

- the budget situation in Orissa has worsened considerably over the period of the project. 

e The Administration recognise that the absence of drugs and other supplies is one of the factors which 
contributes to the lack of credibility of the public health system, resulting in low utilisation of clinics. 

This was among the reasons for their attempt to refocus the project on health management in 1993 

and will remain a priority in any future input they make in Orissa. 

e —_ Local training through district training units established under the project shows more promising signs 
of sustainability. The training is relatively low cost and depends primarily on training teams which have 
developed their own material and are hence strongly committed to its continued use. 


The main lessons from the Orissa project are: 


e realistic targets and an achievable timetable are vital for complex projects 
with many components - especially when local management is weak; 


e health project impacts should be measurable through project specific 
indicators as well as through wider measures of health status; 


e alternative funding channels should be sought if local bureaucracy is likely to 
cause delays; 


e a strong health management capacity in recipient countries is a key 
requirement for effective aid to improve health care which projects must 
address appropriately; 


e a rigorous appraisal of the cost-effectiveness of project inputs is necessary 


before extending their use to a second phase; and 
j 
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e contracted project managers should be required to report showing how 
activity is linked to progress towards objectives, so that the Administration 
can easily identify the key areas needing attention. 


B. St. Stephen’s hospital, Old Delhi 


Project To reduce death and illness among poor women and newborn infants in the St Stephen's 
Objectives Hospital catchment and outreach areas. 

Project - Construction of a 340-bed maternal and child health wing with equipment 

Activities - Development of a community outreach programme in Nand Nagri district 

(1991-94) - Consultancy to examine financing and charging policies of the hospital 


- Training and consultancies for medical staff 
- Establishment of a training resource centre 


Project Total cost: £4.91m (100% Administration funded) 


Cost 


ODA Allocation: £4 .37m building and equipment 
£0.21m technical co-operation 
£0.33m additionally approved in June 1994 


Expenditure: £3.80m by November 1994 
Project St Stephen’s Hospital is responsible for managing the project. ODA contracted the 
Management British Council to co-ordinate project inputs from June 1994 to March 1995 at a cost of 


£35,000. The Council managed the technical co-operation programme for £39,000 
(18% of allocation) 


Targets 


The achievement indicators set by the Administration were sensitive to factors other than project impact 
- for example, the proportion of low birth-weight babies is affected by factors outside the project’s 
influence. 

Other indicators, such as the maternal mortality rate, are not easy to measure. 

The Administration did not secure St Stephen’s agreement to meeting the targets in the project plan. 


Management and reporting 


The Administration did not appoint a project co-ordinator until September 1993, two years after the 
start. St Stephen’s appointed an architect and site supervisors. The British Council arranged for 
consultants to oversee the commissioning of equipment, and set up training visits to and from the UK. 
The Council's buildings adviser visited the project two days a month. 

St Stephen’s submitted monthly reports on progress to the British High Commission in a format 
specified by the Administration. The High Commission reviewed these reports, attended monthly 
building committee meetings and reported to the Administration. 

The Administration’s health adviser expressed concern in July 1991 and January 1993 that project 
management was confused. The Administration’s September 1993 review mission identified a need for 
closer co-ordination, asking the High Commission to oversee the management of all project activities in 
the final year. In June 1994 this work was contracted to the British Council with more detailed terms of 
reference. 


Monitoring 


St Stephen’s monthly reports focused mainly on construction progress and activities at the outreach 
centre; they did not regularly provide updates on progress towards the project objectives. The 
Administration's health adviser visited the hospital and outreach centre regularly but there was limited 
evidence that effective remedial action had resuited on key issues such as financial sustainability. 
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B. St. Stephen’s hospital, Old Delhi 


Monitoring (contd.) 


Much of the Administration’s monitoring focused on the construction programme. Some delays were 
caused by the frequent changes in plans by the architect which in mid-1993 led to a near breakdown in 
working relationships with the building contractor. In March 1994 the Administration stipulated that no 
further plan changes would be accepted. To maintain the pressure for progress, the British Council 
buildings adviser increased his involvement in the project to up to six days a month. 

The Administration did not have a consolidated record of expenditure by component to enable them to 
monitor progress against budget until October 1993. This caused uncertainty in financial management: 
for example, whether funds would be available for additional training of hospital staff. 


Impact 


St Stephen’s has a clear commitment to serving Old Delhi’s poor: 

- 250,000 out-patients a year (40% free of charge); 

- 30,000 in-patients a year (10% free of charge); and 

- the outreach centre in the Nand Nagri slum district provides free treatment for a further 70,000 
patients a year and had surveyed the needs of 53,000 (project target: 100,000); 

The new wing is of a high standard and is likely to enhance the quality and quantity of care available in 

the district. It is not yet in use, so it is not possible at this stage to assess impact in terms of wider 

project objectives. The Administration have not undertaken a rigorous assessment of whether the 

hospital is reaching the poor although a 1990 study concluded that the hospital had the capacity to 

sustain a poverty alleviating policy for poor patients and are at present considering whether and when 

to evaluate the project’s impact. 

St Stephen’s staff have benefited from the training programme, which included study visits to British 

maternity units and in-service training by doctors visiting from Britain. This training has enhanced their 

motivation and led to numerous improvements in medical procedures. 


Sustainability 


The commitment and experience of St Stephen’s provide strong assurance that high quality health care 
for Old Delhi will be sustained. But this is not yet supported by a reliable costing and charging 
methodology designed to ensure that the provision of subsidised and free health care will be financially 
sustainable. 

The need to assess the financial sustainability of St Stephen’s and to identify a suitable charging policy 
was recognised at the outset but not pursued by the Administration until the project was nearing its 
planned completion date. Their project plan had envisaged visits in 1991 and 1993 by a health 
financing consultant to agree a revised fee structure for introduction after the building was complete. 
However they did not invite tenders for the consultancy until August 1993 and no appointment was 
made until December. The consultant visited St Stephen’s in February 1994 and reported in April that 
the hospital “should remain viable”. Although the report recommended ways of improving financial 
accounting and performance and on the apportionment and monitoring of costs, it did not suggest a 
fee structure to maintain services for poor patients. A follow-up visit is planned. 

Future funding for the Nand Nagri outreach centre is also uncertain. When Administration finance 
ceased in March 1995, St Stephen’s had to cover all recurrent costs from its own income or seek other 
donors’ support. Closer integration between the hospital and the outreach centre is essential to 
maintain and expand health care to the target 100,000 slum dwellers. The Administration were to 
address the options as part of a financial consultancy visit undertaken in late 1994. 

Ongoing training for an expanded St Stephen’s staff team was also required and a training centre will 
be established in early 1995. 
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4 The main lessons from the St Stephen’s project are: 


achievement indicators should be closely linked to project activities; 


management and monitoring responsibilities for all project components 
should be clearly allocated from the outset; 


for large construction projects, special attention should be given to pursuing 
key non-construction objectives, such as sustaining health care for the poor; 
and 


the sustainability of health services is more certain when the recipient 
institution is highly motivated and self-sufficient in covering recurrent costs. 


C. Cervical Cancer Research 


Project Wider: to reduce the incidence of cervical cancer through early diagnosis and control 
Objectives Immediate: to understand the role of human papilloma virus infection in cervical cancer 


Project - analysis and follow-up of 12,000 cervical scrapes 
Activities - training of a local scientist in DNA hybridisation techniques for analysing cervical scrapes 
(1990-93) - provision of equipment to the Institute of Cytology and Preventative Oncology 
Project Cost Allocation: £336,439 (£311,000 plus £26,439 management charge) 
Expenditure: British Council figure - £281,023 
ODA figure - £289,012 
Project Carried out by British Council 
Management Management Charges: British Council figure - £37,863 (15% of expenditure) 
ODA figure - £37,443 (15% of expenditure) 
Targets 


No quantified targets were set 


Management and monitoring 


Work awarded to the British Council before the Administration had received an estimate of charges. 
Management charges have exceeded the original estimate without the Administration's agreement. 
Some mis-charging of costs was identified by the National Audit Office and a small discrepancy 
remains. 

Project manager's and consultant's responsibilities were not formally specified and were therefore 
unclear. 

The Maulana Azad medical college failed to collect sufficient scrapes or to record accurately the 
supporting demographic data (only 2,063 fully documented scrapes were collected against a target of 
12,000). As this was not part of the Administration's project the Institute of Cytology and Preventative 
Oncology had little leverage. The Administration did not consider bringing this activity within the scope 
of their project. UK consultants have however accepted that useful results can still be achieved with the 
greatly reduced sample. 

A deep freezer, supplied by the Administration, was kept in a corridor and subjected to extreme 
temperatures and had not been operational for several months. The Institute are making plans to 
relocate the laboratory. 
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5 The main lessons from the Cervical Cancer project are: 


services to be provided by management agents and consultants should be 
clearly defined; 


e project proposals should be reviewed to ensure that there is a clear link 
between activities and objectives; 


e the Administration should exercise more care in charging costs to projects; 
and 


e all possible solutions should be considered when a serious implementation 
problem arises. 
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Appendix 2: Pakistan 


1 Pakistan has a population of 128 million. It is increasing by 3.1 per cent a year - 
one of the highest growth rates in Asia - and is expected to reach 155 million by 
2000. Pakistan’s health indicators are worse than India’s, despite a higher per 
capita income, and only marginally better than those of Bangladesh. Life 
expectancy is 58 years. 


2 The Pakistan Government's policy is to reduce the population growth rate to 
2.6 per cent a year by 2000. The Administration have provided £13 million in 
health and population aid to Pakistan since 1988-89. Their current priorities 
are health sector reform, and ensuring increased provision and use of 
reproductive and other health services by women. 
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A. Population | and Il 


Project No objectives were specifically set for the Administration’s project as it was incorporated in 
Objectives the World Bank’s programme. The wider objectives of this programme were to provide 


support to the Government of Pakistan's Population Welfare Plan aimed at slowing the 
country’s rate of population growth. 


Project - Supply of 91 vehicles (mostly as mobile audio visual units to raise contraception 
Activities awareness) 
(1983-93) - Small innovative" family planning projects (for example, distribution through barbers) 


- Various monitoring and evaluation studies 

- Establishment of two university-based population studies centres 

- Support to 171 non-governmental organisation outlets 

- Supply of injectable contraceptives and contraceptive foam 

Some of these components are part of a larger, £18m, World Bank project. In 1990 a new 
component - District Welfare Activities - was developed to use the unspent part of the 
original grant. This involved general support for family planning activities in five target 
districts and in particular the establishment of eight mobile service units. 


Project Cost Allocation: £4.8m 


Expenditure: £4.45m as at November 1994 


Project Carried out by the Administration in-house. 
Management Cost not known. 


Targets 


The Administration did not set overall project specific objectives though there was a 1989 framework 
for the District Welfare Activities component which set objectives and quantified targets. 


Management and monitoring 


Because of the fragmentary nature of this project the Administration stipulated that a project 
management team should be established but none was set up. 

From the start of the project until November 1985, and again from January 1987 until February 1990 
the Administration had no field management presence. This seriously hindered project progress. 

The Administration do not have accurate records of the expenditure on each component. 

Under some components (for example, vehicles, innovative projects and non-governmental 
organisations) the Administration did not obtain sufficient information to enable them to monitor 
progress. 

As the Administration considered that they did not have the in-house capacity, they contracted the 
Overseas Development Group to provide monitoring services at a cost of £189,569; competitive 
tenders were not sought. 

The London School of Hygiene and Tropical Medicine was contracted to provide assistance under the 
population studies centres component at a cost of £190,632; again no competition was sought. The 
contracted services were mostly provided despite considerable difficulties posed by local bureaucracy 
and political turmoil. The Administration did not take action when the School, mainly because of 
shootings in Karachi, failed to follow all the agreed reporting arrangements. | 
Progress reports did not always show progress clearly against the targets. 

The vehicle supplier was chosen despite its limited network in Pakistan. The vehicles were made to 
specification but on receipt of the first consignment the Government of Pakistan requested window 
conversions for 75 of them and this was carried out by the supplier at a cost of £22,500. A 1988 
assessment found that the structure of the vehicles had suffered major damage as a direct result of the 
conversions and many were off the road. The supplier provided free repair kits and paid for their 
shipment to Pakistan. The Administration paid £16,700 for the repairs to be carried out. 
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A. Population | and II 


Impact 


The project's achievements need to be seen in the context of wider political difficulties. During the 
1980s, attempts to integrate population activities within the health infrastructure failed and the 
population programme suffered from over-centralisation. The Administration had to work against 
substantial political and bureaucratic obstacles. 

The Administration have assessed the impact of some components well while on others, such as 
innovative projects, there is an absence of material. They did not assess overall impact until 1994 when 
a draft project completion report was produced; this had yet to be finalised in October 1994. 

Overall the project has had no more than a limited impact. The Administration consider the project has 
given them field credibility through which they have become an influential voice in national population 
policy. 

Although there is limited information, there is some evidence that minimal use was made of the 
vehicles: the average monthly number of audio-visual shows per vehicle was 6.4. One of the problems 
was that the rationale behind this component was inconsistent with the original proposal - which 
Stated that awareness of contraception was already high and the need was to make services available. 
The Administration supplied £450,000 of contraceptive foam and £500,000 of injectable 
contraceptives. A 1990 study considered that the Norigest supplies were cost-effective and ethically 
respectable and that the distribution system was efficient. The injectables were of particular benefit to 
the poor and to women (who could control their use alone). The study considered the supply of foam 
to be less effective. Although it made an important contribution towards ensuring that a range of 
contraceptives were available, it was too expensive for the poor. 

Two centres for conducting demographic research were established and training provided contributed 
to an increase in demographic expertise in Pakistan. A 1988 consultant's report found that the 
standard of research was low but also considered that there was no chance that this component could 
succeed in its two to three year life. The centres were not successful and the Administration ceased 
funding. 

There is little evidence on which to assess the impact of support for non-governmental organisations. 
In general the Administration consider they provided a good service although their approaches are 
now seen as conventional and not necessarily the best way of promoting family planning. 

Apart from eight vehicles (mobile service units) supplied under the District Welfare component, 
funding did not reach the target districts - though it did provide general budgetary support to the 
Ministry of Population and Welfare. Mobile service units were the most successful element and are 
now used more widely across Pakistan. The immediate impact on family planning services was only 
marginal: the total number of clients in target districts increased by 10% over the component's life. 


Sustainability 


There was a lack of local ownership of some project activities. 

The Administration funded the population centres until 1990 when the Ministry of Education undertook 
to take over recurrent cost responsibility. However this was not made a condition of the grant. The 
Ministry place little value on the centres and have not taken over their funding. One centre ceased to 
function entirely in July 1993 and the other is desperate for funds. 

The Government of Pakistan insisted that non-governmental bodies were funded through a central 
Co-ordinating Council, under their control. A 1994 London School of Hygiene and Tropical Medicine 
report criticised donors for funding through this body as there had been clear evidence since 1988 that 
it lacked the capacity to use funds effectively. Funding of these outlets finished, as planned, in June 
1993. Since then some 35 of the 171 outlets funded have folded. The Administration did not consider 
their sustainability at the appraisal stage. 
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3 The main lessons from Population I and II are: 


the need to establish a stronger field management presence for complex 
projects; 


project frameworks should be drawn up with a coherent set of inputs 
contributing to a clear overall purpose; 


the Administration should ensure that they receive adequate information on 
which to assess project progress; 


project expenditure should be analysed by component to facilitate financial 
monitoring and enable value for money to be assessed; 


the Administration should be firmer in asserting their rights as purchasers of 
goods and services; 


early assurances should be obtained that host countries will take over 
recurrent cost responsibility after project completion; and 


funding mechanisms should ensure that all project money actually goes to 
the project activities rather than providing general population support. 


B. Reproductive Health Services 


Project - Contribute to the Pakistan Government’s objective of reducing the population growth rate 
Objectives from 2.75% (1987) to 1.66% (1991) 


- Reduce maternal and infant mortality 
- Ensure adequate availability of reproductive health services throughout the country 


Project - Renovation/construction of 45 reproductive health service “A” centres 
Activities - Funding of contraceptive surgery 
(1987-93) - Supply of medical equipment 


- Training of doctors and paramedical staff in contraceptive surgery techniques 


Project Total cost: $3.46m (also funded by UN Population Fund and Pakistan Government) 
Cost ODA allocation: | £2.5m ($3.12m at 1987 exchange rate) 


ODA expenditure: £1.77m as at November 1994 


Project ODA were supporting a project set up and managed by the UN Population Fund, which 
Management charged them a management fee of 5% of their contribution 


Targets 


Achievement indicators set by the Population Fund related to construction and the number of 
sterilisations. 

Targets for sterilisations were unrealistically high: the Fund now considers that qualitative targets 
would have been a better management tool and that quantitative targets run the risk of encouraging 
health workers to coerce women into surgery. 

The Fund did not set indicators to show when “adequate” availability of reproductive health centres 
had been achieved, or to show progress towards the aim of reducing maternal and infant mortality. 


/ 
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B. Reproductive Health Services 


Management and reporting 


The Population Fund managed the project in co-operation with the Pakistan Government and undertook 
to provide six-monthly progress reports to the Administration. 

Annual reviews took place, in which the Administration were invited to participate. A mid-term review 
was also planned but not carried out. 

The Fund agreed to prepare a final evaluation report but to date this has not been produced. 


Monitoring 


The National Audit Office found three progress reports covering the period 1990-92. The Ministry of 
Population and Welfare produced a final progress report in December 1993 covering the previous 
seven years. These reports provided basic statistics on expenditure, centres built, training and 
contraceptive treatment but included only limited analysis of progress towards project objectives. 

The Administration’s health advisers also monitored progress by visits and participation in annual 
reviews. The British High Commission followed up specific points locally and from February 1990 had 
professional management oversight of the project. 

By 1992 the cost of building each health centre had doubled to Rs700,000 (£15,900). The 
Administration did not provide additional funding but asked the Population Fund to set quarterly 
benchmarks for construction and report on progress. However, the Fund’s progress reports contained 
no information about such benchmarks or of additional monitoring of construction work. 


Impact 


The construction target of 45 reproductive health centres was met, bringing the total to 80 
country-wide. While total figures for contraceptive services provided are available, the impact of the 
new centres is not yet known. The centres are attached to larger general hospitals and therefore are 
conveniently located to serve urban women. However, a significant proportion of contraceptive services 
are provided at mobile camps held in villages. Rural women find this more convenient and the service 
could be expanded if more staff were available. 

The project met 48% of the target 917,000 sterilisations between 1987 and 1993. The Population Fund 
identified the reasons for this under-achievement as low training standards and weak motivation and 
counselling. The centres were also understaffed during the first three years, when a Government ban on 
recruitment was in place. The situation has since improved and about 90% of the required staff are now 
in post. 

The Administration estimate that the sterilisations averted some 590,000 births over the life of the 
project and consider this to be a cost-effective contribution to containing Pakistan's population growth. 
Nonetheless the country’s annual growth rate is 3.1% - still well above the Government's target of 2.6%. 
The number of doctors and paramedical staff trained in sterilisation techniques exceeded project targets 
but the Fund considers that the standards of training were not sufficiently high. Impact has not yet been 
assessed 


Sustainability 


The Administration did not address the issue of sustainability when agreeing to contribute to this 
project. However the Fund remains committed to strengthening reproductive health services and 
training. 

The Administration consider that the social action plan agreed between the Pakistan Government and 
the World Bank, and co-funded by the Administration, will provide some guarantee of adequate staffing 
and recurrent cost provision for these services. 
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4 The main lessons from the Reproductive Health Services project are: 


e realistic targets should be set which are directly related to the project inputs; 


and 


e when an independent implementing agency is used, the Administration 


should set out a clear reporting format which links activity to progress 
towards project objectives. 


C. Asian Development Bank III 


Project To improve health care and essential support services in Pakistan and particularly in 
Objectives Balochistan and the North West Frontier Province (NWFP). 


Project - Establishment of specialist training courses for nurses (Balochistan and NWFP) 
Activities - Establishment of an unspecified number of repair and maintenance workshops for medical 
(1991-96) equipment (Balochistan and the North West Frontier Province) 


- Introduction of strategic planning to federal and all provincial health ministries 


Project Cost Allocation: £3.04m (part of a £23m Asian Development Bank project) 


Expenditure: £1.70m as at November 1994 


Project Carried out by the British Council 
Management Management charges: £459,050 (15% of allocation) 


Targets 

The indicator for monitoring the achievement of the wider objective was too broad to measure impact. No 
quantified targets have been set to measure the achievement of immediate objectives although some targets 
have been set for project outputs. 


Management and monitoring 


In 1991 the Administration expressed concern at the size of the Council’s proposed management 
charge and stated that the fee should be reviewed after one year. No such review was conducted. 
Between September 1991 and October 1992 the Administration expressed dissatisfaction with the 
management services but did not consider alternative ways of managing the project and made no 
changes to the management fee. 

The Administration did not specify a format for progress reports. Before April 1994 these reports 
focused on inputs and did not relate progress directly to the project objectives. 

Monitoring visits identified a number of implementation problems - for example in North West Frontier 
Province, the repair and maintenance component has been held back because the provincial 
government has not sanctioned posts in the central workshop. The Administration have been 
monitoring the situation regularly and have put pressure on the provincial health ministry to take 
effective action. In May 1994 their inputs were suspended and they have since received assurances 
that the posts will be established. ; 
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C. Asian Development Bank III 


Impact 

e The first specialist training courses for nurses began in the North West Frontier Province in March 
1993 and the nursing consultant judged the courses successful and pointed to an increase in 
knowledge base, practical skills and self-confidence among the students. 

e There have been difficulties in recruiting nurses for courses because of a reluctance by the authorities 
to release nurses for training. There is also concern that nurses who complete these courses may not 
be posted to jobs where their new skills will be used, although so far most have been appropriately 
posted. 

e By May 1994, two nurse tutors had returned to Pakistan following training in the UK. They found their 
training broadly useful and relevant. English medical books supplied to the nursing colleges in 
Peshawar and Quetta are proving of limited use to the students, who have only a limited knowledge of 
English. 

e —_ The Administration undertook to supply a minibus to Peshawar nursing college to enable students to 
attend practical demonstrations. Its purchase has, however, been delayed since 1992. 

e _ The Administration consider the planning component to have been the most successful part of the 
project and for this reason they extended it to the remaining two provinces. Elements of it have been 
incorporated in Pakistan’s social action programme and three of the four provinces have already made 
good progress in adopting strategic planning. The federal: planning cell, where two officers have so far 
received UK training, has encountered difficulties in utilising their trainees’ new skills. One has been 
transferred out of the cell. 

e The repair and maintenance component has yet to make any headway in North West Frontier Province. 


Sustainability 

e Sustainability of the nursing component will partly depend on permanent teaching posts being 
established by the provincial health ministries at nursing colleges. 

e There is uncertainty over the commitment of federal and provincial governments to continue 
supporting the planning cells, some of which do not have regular budgets or permanent posts. The 
National Audit Office visited the cell in North West Frontier Province and found its accommodation to 
be unsatisfactory and cramped. 


The main lessons from the Asian Development Bank III project are: 
e targets should be clear, specific and measurable; 


e arigorous assessment should be made of the value for money associated 
with the different possible management arrangements; 


e the Administration should ensure that effective mechanisms are in place to 
monitor contractor performance; 


e the need for early and decisive action where there is a clear lack of 
commitment to the project from the implementing body; and 


e the need to encourage recipient countries to employ trainees in a position to 
apply their new skills. 
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D. Population Ill 


Project To reduce the gap between wanted and actual family size (the total fertility rate) by 
Objective allowing women and men to exercise increased reproductive choice. 

Project - Supply of contraceptives 

Activities - Institutional development of the Ministry of Population and Welfare 

(1994-97) - Various operational research studies on the family planning programme 


- A National Fertility and Family Planning Study in 1996-97 

- A contraceptive distribution study 

The project was approved in March 1994 and the Government of Pakistan formally 
accepted the grant on 31 July 1994. 


Project Cost Allocation: £9.045m 
Expenditure: £1.0m as at November 1994 
Project Management and procurement under the contraceptive supply component to be carried 


Management out by the UNFPA at a cost of £380,000. Other project management to be carried out 


in-house by the Administration. 


Plans for implementation 


£7.6m of the grant is intended to provide part of the national requirement for contraceptives but, at 
present, future requirement figures are not reliable. The Population Fund is addressing this and the 
Administration will release funds only on receipt of reliable reports on the stock position, the future 
requirement and the level of expenditure incurred. 

An Administration-funded study began in 1995 to examine improvements to the distribution system 
being made under the UNFPA project. 

Apart from a research study, planned to start in 1994, other components have been specified only in 
outline. 

In anumber of cases the Administration have not specified their activities in detail because they are 
co-ordinating their inputs with other donors’ activities linked to the World Bank’s multi-sector Social 
Action Programme. The Administration have developed regular contacts with other donors to 
co-ordinate action. 


Targets 


The Administration have set indicators for project outputs which will provide a basis for monitoring 
project progress; most of these have been quantified. In February 1995 the reporting format was being 
drawn up. 

The Administration aim to measure the increased use of reproductive choice by monitoring the 
reduction in the gap between wanted and actual family size; specific targets have not, however, been 
set. 


Management and monitoring 


The Administration have nominated the members of a project management team which will be the key 
decision-making body and this has started to hold ad hoc meetings. 

The Administration’s Health and Population Office in Islamabad is well-placed to carry out field 
management. The Administration have yet to decide what proportion of their staff time will need to be 
spent on the project. 

Once activity begins, the Administration intend to carry out financial monitoring by setting up separate 
expenditure codes for each of the components. So far three codes have been set up for the UK 
pre-implementation costs, which by June 1994 stood at £111,000. Not all implementation costs have 
been charged to the project because locally incurred costs have been allocated to a separate cost code. 
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The main lessons from the Population III project are: 


e the first phase activities should be fully designed, together with associated 
targets and indicators; 


e close monitoring of the country’s contraceptive requirements is essential; and 


e effective distribution systems need to be established to maximise potential 
impact. 
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Appendix 3: Kenya 


1 Kenya’s population of 26 million is increasing by 3.3 per cent a year and despite 
its declining population growth rate, is expected to reach 35 million by 2000. 
There is a serious AIDS epidemic, with HIV prevalence estimated at least at 
5.6 per cent, and the number who may be infected with the disease by 2000 is 
estimated to be 1.6 million. Life expectancy in Kenya is 61 years. 


2 Kenya’s health policy is to undertake health sector reform designed to improve 
the management and implementation of health services and to increase staffing 
efficiency. The Administration have provided £14.2 million in bilateral health 
and population aid to Kenya since 1988-89. Their current priorities are to 
support health sector reforms and provide for the immediate needs of Kenya’s 
reproductive health services. 
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A. Isiolo Hospital and Immediate Support 


Project - Build and equip a 160 bed hospital and 60 staff housing units at Isiolo by October 1988 
Objectives - Improve the quality of out-patient health care in the Isiolo district 


- Provide improved and sustainable surgical referral services at Isiolo 


Project - Construction of a 160 bed district hospital in Isiolo 

Activities - Construction of 60 associated staff housing units for hospital employees 
(1986-90 and = - Provide a maintenance engineer and a surgeon for three years 

1990-94) - Supply an ambulance 


- Isiolo Hospital phase II immediate support for medical supplies (£0.57m) 


ProjectCost —_Total cost: £3.25m (2.8m construction costs and £385,000 technical co-operation) 


£569,000 Isiolo hospital phase || immediate support 
Expenditure: £3.8m (£0.52m technical co-operation) - by November 1994 


Project Carried out by TRIAD (Architects/planners) and Crown Agents (procurement of consumables) 
Management Management charges: £394,000 (14% of allocation) 


Project history The contract was awarded to Gina Ratna for Ksh77.6m (£2.53m) after some initial 


difficulties in the tender selection process. The hospital was formally opened in January 
1990, after which the Administration provided drugs and medical supplies. 


Targets 


To construct a 160 bed hospital arid 60 housing units by October 1986 ai a cost of £3.25m. 

No other measurable indicators were set. There was no clear definition of the catchment area or the 
hospital's relationship to other health facilities attempted. And there was no assessment of its likely 
impact on recurrent costs to the district, or limits suggested. 


Management and monitoring 


Architects were engaged in 1986 to design the hospital and supervise the work. 

Progress reports were provided in a format specified by the Administration’s architectural adviser; 
these focused on construction progress and problems encountered. 

AMREF were appointed to oversee the commission of the hospital equipment and stores and Crown 
Agents to monitor the storage and distribution of consumables. 

Most of the Kenya Government payments were late; this delayed many of the UK payments. 


Implementation 


The Administration approved three time extensions because of problems caused by delays in making 
payments to the contractor and in obtaining import licences. 

The hospital was completed in December 1988 and patients were first admitted in February 1989. 
The final construction cost of Ksh86.6m was Ksh9m in excess of the Ksh budget but was within 
original sterling budget, due to favourable changes in the exchange rate. 

The maintenance engineer and surgeon provided expertise to set up a hospital maintenance support 
unit and establish a surgical referral service. 
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A. Isiolo Hospital and Immediate Support 


Impact 


It is difficult to assess impact because of the Administration's difficulty in setting specific targets at the 
project design stage. 

No data were produced on quality of service, referral function or target throughput of services to 
patients. 

The hospital was completed, equipped and brought into use with only minor delay. 5 

A much needed facility and a wide range of medical services have been provided to the district. 
Hospital workload statistics show an increase in the number of in-patients treated and major surgical 
cases undertaken. Minor surgical cases have, however, significantly decreased. 

The need for adequate stores accounting systems was recommended but supervision and systems 
have not improved - a quantity of insulin cannot be accounted for and the police are investigating the 
discrepancy. The National Audit Office also observed some excess stock of UK funded supplies and a 
further stock discrepancy. 

The vehicle workshop was initially by-passed and fleet repairs undertaken at a garage in Isiolo. 


Sustainability 


The hospital displays a good general appearance, is clean, well equipped and operational. Some 
structural cracking has occurred and this is being investigated to assess its seriousness. 

Except for the vehicle maintenance workshop, all the facilities appeared to be utilised. 

Maintenance engineer assistance was conditional on a counterpart being employed to study 
appropriate techniques but he could not undertake his training role for 15 months until counterpart 
was appointed. 

The surgeon was provided on condition that he would be replaced immediately after his contract was 
completed; it took a year to appoint a replacement surgeon. Efforts are now being made to find another 
replacement. 

The maintenance department was functional and preventative maintenance system continues to 
operate. Except for two items, all the equipment was operational and appeared to be utilised. 

The hospital recurrent budget (excluding staff costs) is to increase by 27% over two years. The Kenyan 
Government have given assurances that they will make adequate staffing and recurrent cost provisions. 
Staff numbers and costs have increased since the hospital opened. The complement appears high - an 
analysis of the recommended staffing levels against current levels by the National Audit Office 
suggested a doubling. The Administration informed the Ministry of Health of the high staffing levels 
but they had not acted on this advice. 

The old hospital, which Isiolo Hospital was meant to replace, is being refurbished as a 99 bed 
tuberculosis unit. Its operating cost (Ksh0.5m) will add pressure on the existing hospital recurrent 
budget. 


The main lessons from the Isiolo project are: 


the requirement for a hospital facility should be based on an assessment of 
need, including defining the target population; 


the impact of the hospital on recurrent costs should be assessed at the outset; 
setting up a preventative maintenance system is helpful; 


consideration of immediate support requirements is essential for the effective 
running of a new facility; and 


yf 
the need for effective control over stocks. 
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Wider Project —- To reduce the rate of population growth through greater awareness of family planning 
Objectives and better services 


Immediate - To increase the acceptance of, and demand for, family planning services through an 
Project information, education and communication programme 
Objectives - To increase the availability, accessibility and quality of family planning services provided 


by the Government of Kenya and non-governmental organisations 
- To strengthen the capacity of the National Council for Population and Development to 
plan, programme, finance, co-ordinate and monitor programmes 


Project - Expand family planning services in five non-governmental organisations in Kenya 
Activities - Support new initiatives by non-governmental organisations 
(1988-94) - Study long-term alternative non-governmental organisational funding mechanisms 


- Train National Council for Population and Development staff 


Project Cost Total cost: $28.3m (World Bank $12.2m, Kenya $2.7m, Norway $5.0m) 
UK contribution: £4.6m ($8.4m) and £0.3m (increased to £495,000) technical 
co-operation 
UK expenditure: | £4.17m (£490,000 technical co-operation) - by November 1994 


Project Managed by the Kenya National Council for Population and Development and the 
Management Administration’s Development Division.Procurement agents were Crown Agents. The 
Administration also funded a World Bank project manager and co-ordinator. 

Targets 

e —_ The project framework set generally measurable targets at the output level, although quantifiable 
indicators of impact were less clearly established. 

e The five indigenous non-governmental organisations funded under the UK component set specific 
objectives but quantifiable indicators were not set. 


Management and reporting 

e The National Council for Population and Development have responsibility for co-ordinating the 
monitoring and implementation of population programmes. 

e The Administration funded a population project co-ordinator who was seconded to the local World 
Bank office to manage and oversee the implementation of Population III and IV. 

e The project was extended one year from June 1993 following implementation and disbursement 
problems, the World Bank components of this project are some three years behind schedule. 

e —_ The Administration reimbursed the Council through the Kenyan Treasury for advances made to the five 
participating non-governmental organisations. The procedures for transferring funds were, however, 
recognised by the Administration to be inadequate to ensure the timely receipt of funds by 
implementing agencies and attempts to get the Government of Kenya to change the procedures failed. 

e There was little activity in the first year, except continuous revision of the five agency plans. 


Monitoring 

e Participating agencies were required to produce quarterly progress reports in a format that was 
changed by the Council on a number of occasions. These gave a clear picture of the various activities 
being undertaken but did not measure progress against targets set or achievement of project 


objectives. 

e —_|mproved monitoring could have been achieved if the Council had produced summarised progress 
reports. 

e The Administration participated in all eight review missions undertaken by the World Bank during 1989 
and 1994. 
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Monitoring (contd) 


The project co-ordinator did not perform to a satisfactory standard due partly to unclear terms of 
reference and partly to the difficulties of reporting to both the World Bank and the Administration. This 
vital co-ordination role was insufficiently supervised in-country by the World Bank and key project 
activities were not sufficiently progressed. The Administration did not therefore renew the post beyond 
December 1993. 

The Administration insisted on appointing an architect to review the design for Nyeri and Kisumu 
clinics; the proposed re-design reduced construction costs by Ksh6m (£65,000). 

The Administration suspended disbursements in 1992 when the Government of Kenya failed to 
provide audit reports, similar action was not however taken by the World Bank. 


Impact 


The impact of the different components of this project have been variable and difficult to measure 
because of the lack of an effective monitoring system, inadequate/no assessments and the absence of 
measurable indicators. 

Delays in funding slowed the project and reduced the final impact. 

Assessments have been carried out by only three of the non-governmental agencies, despite the 
Administration’s efforts to encourage the Council to ensure that suitable arrangements were made. 
The assessments found that service delivery had been the most successful element while the 
information, education and communication components were only partly successful. 

The Administration considered only one assessment to be to a satisfactory standard. 

The Kenyan population programme has made considerable progress over the last few years. The 1993 
demographic and health survey indicates increased use of modern contraceptives, reduction in the 
desired size of families and almost a 100% awareness of modern family planning methods. This 
project can claim some credit for these improvements but it is difficult to quantify or attribute the 
individual components’ contributions. 

More clients are using family planning services but the unmet demand for these services has not 
declined. Almost the same number of people in 1989 and 1993 stated that they wanted to use the 
services but for a variety of reasons, were constrained from doing so. 


(a) Family Planning Association of Kenya component 


(Allocation £1.533m: Expenditure by November 1994 £1.76m) 
A new Clinic at Nyeri, built at a cost of Ksh9.55m (£104,000) and opened in January 1994, has 
significantly improved facilities but client attendance initially dropped by 7% because of relocation. It 
has now recovered and is reporting figures above pre-relocation levels. 
A new clinic in Kisumu was completed by the end of 1994 at an estimated cost of Ksh14m (£155,000). 
Both new clinics replaced outdated poorly equipped clinics. 
Service statistics for all the Association's clinics between 1989 and 1993 show a 17% reduction in the 
Static clinics’ performance and a reduction of 8% in outreach performance. 
The Association considered that its performance had actually improved and said that, in the above 
figures, it had double counted client throughput in the years up to 1992. 
Population Ill has increased the operating costs of the Nyeri clinic but data were not available on the 
overall level of the increased recurrent costs of the rehabilitated clinics 


(b) Christian Health Association of Kenya component 


(Allocation £1.221m: Expenditure by November 1994 £1.2m) 
The Administration, in collaboration with a Dutch donor, have assisted the Association in rehabilitating 
31 clinics at an average cost of Ksh722,000 (£8,000) - of which the Administration contributed 52%. 
For Administration-funded clinics more clients were assisted for the child welfare, ante-natal, curative 
and family planning services - an overall increase of 93,400 (22.5%) in clients assisted. 
jf 
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Christian Health Association of Kenya component (contd) 

e The nearly completed rehabilitated clinic at Ngecha visited by the National Audit Office will provide 
substantially more space and facilities than the old, cramped, dispensary. 

e —_ Under the youth training element, a total of 3,185 youth educators have been trained, who have in turn 
provided youth training to up to 230,000 young people each year. 

e The impact of this training has not yet been satisfactorily assessed. A consultancy assessment was 
undertaken but it was not supervised and the output proved unacceptable. 


(c) National Council of Churches of Kenya component 
(Allocation £0.741m: Expenditure by November 1994 £0.73m) 

e —__ The impact of this component is difficult to measure. Some 3,100 volunteers and motivators have 
been trained under the Council’s family life education programme but more than half have since left the 
project. 

e _ Part way through the project the Council informed the National Council for Population & Development 
that it had changed its objectives by excluding the community-based distribution of contraceptives but 
no action was taken. 

e _Aninternal assessment was carried out by the Council but it proved entirely inadequate, as the Council 
did not have the technical expertise to undertake the work. 


(d) Kenya National Union of Teachers component 
(Allocation £330,000: Expenditure by November 1994 £200,000) 

e 38seminars, involving some 1,700 teachers, were undertaken by the Union between 1988 and 1992 
to make teachers aware of the problems involved in family planning and population expansion. No 
formal impact assessment has been undertaken. 

e —_ Both the Union and the Administration felt the programme was extremely weak and ceased funding in 
1993. 

e Aminibus provided to the Union in 1990 to transport seminar participants has not been used. Some 
attempts have been made to find an alternative use for the vehicle but as at July 1994 it remained 
unused. 


e 
(e) Kenya Catholic Secretariat component 
(Allocation £180,000: Expenditure by November 1994 £220,000) 

e The family life programme encourages the use of natural family planning and has been operational for 
12 years but as yet no formal impact assessment has been undertaken. 

e — Since 1982 over 1,400 teachers have been trained including 250 in the last four years - of which 1,000 
and 200 respectively remain active. Some 4,140 clients enrolled for natural family planning training in 
the four years since April 1988. 

e Anevaluation is being planned to establish the impact, user satisfaction, cost effectiveness (compared 
with artificial methods) and the benefits attained. 

e The Administration propose to continue to provide modest funding for natural family planning 
programmes to ensure that a full range of family planning methods remains generally available. 


(f) Innovative projects component 
(Allocation £271,000: Expenditure by November 1994 £44,000) 
e _Little activity has been undertaken on innovative projects. 
e Problems were encountered in approving projects submitted by the agencies which were acceptable to 
the Administration and the National Council for Population and Development. 
e The Council failed to encourage and develop new non-governmental organisations and private sector 
providers as only 25% of the funds have been allocated. 
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4 The main lesson from Population III are: 


e adequate procedures for disbursing funds should be established at the 
project outset; 


e if the management capacity of the monitoring organisation is not assured, 
consideration should be given to undertaking additional monitoring reviews; 


e management and monitoring responsibility for all project components should 
be clearly specified at the outset; and 


e measurable targets should be set directly related to the project’s objectives. 
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C. Family Planning Services 


Wider Project —- To assist the Government of Kenya to reduce the population growth rate by lowering 
Objectives the total fertility rate. 


Immediate - To increase the demand for family planning 

Project - To increase the number of functional family planning clinics 

Objectives 

Project - Supply three million doses of Depo-Provera (with disposable syringe and needle) 
Activities - Supply 40 family planning equipment kits to government/non-governmental 
(1991-94) organisations 


- Supply 450 Intra Uterine Contraceptive Device insertion clinic kits 


Project Cost Total cost: £1.97m (plus £30,000 technical co-operation funding) 
Expenditure: £1.96m as at November 1994 (technical co-operation £14,000) 
Project Carried out by Development Division. Procurement undertaken by Crown Agents 


Management Monitoring cost: £14,000 (0.7% of allocation). Procurement fees: £109,000 


Targets 
e _ In addition to quantities of contraceptives to be delivered the Administration set a number of target 
indicators. 


e The main target was to achieve 750,000 couple-years of family planning protection. 

e Specific targets for this project were set before the significant increase in the uptake of family planning 
services which surpassed the Government of Kenya projections; as a result targets were not 
sufficiently robust. 


Project management 
e _ Project approved in April 1991 and the Development Division in Nairobi was given management 
responsibility. 


e Crown Agents were appointed as procurement agents without competitive tendering. 
e Options (formerly Marie Stopes Consultancy) monitored the distribution of supplies. 


Monitoring 

e The Administration monitored the project by attending monthly contraceptive logistics committee 
meetings (a multi-donor/Ministry of Health co-ordination group), periodic visits to the family planning 
logistics management project and the Ministry of Health family planning stores, and reviewing 
progress on contraceptive deliveries. 

e 1.3mdoses of Depo-Provera were ordered in September 1991. They arrived in Kenya in November 
1991 but took six months to reach the medical stores control unit. 16 clinic kits and 180 intra-uterine 
device insertion kits were also ordered in September 1991; these arrived in April 1992 and took 
five months to reach their destination. 

e Delays occurred at Embakasi port because handling charges were not paid - the Administration did not 
fund these charges as itis their policy not to pay for in-country costs that involve inter-departmental 
charges and taxes. The other two tranches of supplies suffered the same problem and delays occurred 
in distributing supplies. 

e Options carried out a monitoring visit in October 1992 to assess progress towards project objectives. 
The evaluation confirmed that the contraceptives had been delivered to the central/district stores and 
were being distributed to service delivery points. 

e — Insome minor instances supplies were not fully delivered or utilised. 90% of service delivery points 
received their Depo-Provera with needles and syringes. 
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C. Family Planning Services 


Monitoring (contd) 


Options reviewed the clinical standards applied to the use of Depo-Provera and found that adequate 
medical standards were being maintained. It also found that some non-consumable equipment had 
never been used. As a result the clinical equipment component was dropped and disposable gloves 
were provided with the third tranche. 

The final consignment of UK-funded Depo-Provera had been at Embakasi port for five months at the 
time of the National Audit Office visit in June 1994. Stocks were then very low and unless urgent action 
was taken there was a high risk that stocks would run out. 

2.8m doses of Depo-Provera have been delivered since April 1992. The final UK-funded consignment 
of 446,400 doses represented sufficient supplies for Kenya up to the end of 1994. 


Impact 


The project has been a major success in increasing the use of injectable contraceptives in Kenya and 
has directly increased the number of functional family planning clinics by 23. 

The couple protection years achieved from the UK-funded Depo-Provera is assessed at 725,000 (an 
average of 260,000 a year). This is a significant increase over the figure for 1990 of 172,000. 

The Administration’s consultants considered Depo-Provera to be a vital component in the Kenya family 
planning programme. It is popular with women in Kenya who are offered a full choice of contraceptive 
methods. 

The injectable method is generally considered the most effective reversible method of contraception. 
Consumer demand for Depo-Provera is increasing where high continuation rates (80% over 

12 months) have been recorded. 

During the period the Administration have been funding the supply of Depo-Provera, the contraceptive 
prevalence rate for injectables has also increased from 3.3% to 7.2%. 

The level of contraceptive use has almost doubled in Kenya in the past decade - from 17% of married 
women in 1984 to 33% in 1993. The use of injectable contraceptives has increased from 3.7% in 1991 
to 7% in 1993. 


Sustainability 


Less than 5% of the current contraceptive supply in Kenya is funded from local Kenyan sources but 
limited consideration has been given to reducing the Kenyan Government's reliance on donor support. 
More than 95% of the Depo-Provera being supplied to Kenya is now funded by the Administration, 
against a planned provision of 60%. Planned UNFPA supplies were changed to latex gloves at the 
request of the Government of Kenya. Donor meetings are being planned to determine whether 
alternative supplies can be funded by other donors or by the Kenyan government. 

Despite the significant increase in the supply of Depo-Provera from UK aid to Kenya, the National Audit 
Office noted that the Family Planning Association of Kenya was obtaining its own supplies. It was also 
considering setting up a separate contraceptive distribution network because of previous poor 
performance of the Government network. 

The Association however accepted that Government supplies had significantly improved in recent 
years and that it would be more effective to use this network to obtain future supplies. 


5 The main lessons from the family planning project are: 


e injectable contraceptives are now the main method of contraception in Kenya 
and could prove useful in other developing countries; 


e close monitoring of the country’s contraceptive requirements is essential; 


f 
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effective remedial action can be achieved from well directed mid-term 
reviews; 


all major constraints to the successful implementation of a project should be 
highlighted at the project appraisal stage; 


targets should be realistic and measurable during a project’s life; 
the problems in quantitatively measuring project impact, or the goals set; and 


opportunities for developing indigenous suppliers should be sought. 
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Appendix 4: Zimbabwe 


1 Zimbabwe's population of 10.9 million is increasing by 2.8 per cent a year and, 
despite having the highest contraceptive prevalence rate in Africa, is expected to 
reach 13.2 million by 2000. There is a serious AIDS epidemic, with HIV 
prevalence estimated at least at nine per cent; the number who may be infected 
with the disease by 2000 is estimated at two million. Life expectancy is 58 years. 


2 Zimbabwe’s health policy is to promote health and quality of life by providing 
appropriate health care in the community and hospital services. The 
Administration have provided £4.9 million in bilateral health and population aid 
to Zimbabwe since 1988-89. Their current priorities are health management 
strengthening, contraceptive supply and research into malaria control. 
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A. Health Management Strengthening 


Wider Project = - To obtain optimum use of limited health resources and to create sustainable 


Objectives management systems and procedures 

Immediate - To give health sector staff access to relevant information to manage resources more 
Project effectively by creating appropriate management systems. 

Objectives - To mobilise increased resources for the sector 

Project - Technical advisers on health management and financial planning 

Activities - Short-term consultancy (Organisation Development Consultant) 

(1991-95) - Link with UK Health Authority providing regular short-term assistance (NICARE) 


- Local consultancy and training 
- Supply of books, equipment and computers 


Project Cost Total cost: £2.063m parallel financing with the World Bank 
UK expenditure: £957,104 as at November 1994 
Project Carried out by Development Division. Procurement initially carried out by Hundhill and 


Management later by Balfour Williamson. Procurement fees: £13,860 (5% of goods) 


Project History The project was drawn up in liaison with the Zimbabwe Ministry of Health and the World 
Bank in the light of experience with the previous Family Health Project |. Under that 
project the UK provided support for personnel and general management to eight pilot 
districts to improve the Government's institutional capacity to plan, manage and evaluate 
health services. 

Targets 

e — The project was not fully designed when it was approved and the project proposal was too open-ended. 

e The project framework did not specify precise indicators for the immediate objectives, leaving them to 

be defined in baseline studies carried out during the first two months of the project. The baseline 
Studies were not completed until 1993 and measurable indicators have not been incorporated into the 
project framework. 

e Other areas where the project design did not adequately address implementation issues include: 

- no prioritising individual components or inputs 

- failure to establish a robust disbursement mechanism 

- not determining the Ministry of Health and Child Walfare’s contributions and obligations 

- not setting targets for budget allocations to shift expenditure towards primary health care 

- failure to determine the requirements for computers, books or local consultancies at an early stage. 


Monitoring 
e The Administration’s health and population adviser in Malawi had managerial and supervisory 
responsibility. 


e — Six-monthly meetings were proposed to monitor the progress of the project with annual reviews 
attended by all project participants. In the event six-monthly reviews were undertaken which were 
attended by the programme director, all component leaders, technical co-operation officers, the Health 
Authority consultants (NICARE), British High Commission staff and advisers from the Development 
Division. 

e The level of review appears to be excessive in relation to the spend, although the Administration 
consider it appropriate at the time to ensure project decisions were taken at the highest level. 

e Some duplication of work arose with NICARE, the ODC consultant and the Ministry of Health 
producing reports. 

e The format of consultants’ reports differed and was confusing; the Administration could have specified 
a format designed to draw out progress towards achieving objectives. NICARE proposed revised 
reporting arrangements in early 1995 which would explicitly show targets, priorities and controls. 
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A. Health Management Strengthening 


Project implementation 

e _— Insufficient attention was given to improving the quality of the early project implementation, which has 
been particularly slow: there have been long delays in the appointment of the two advisers; nearly two 
years delay in procuring equipment and supplies, some delays in the NICARE input (after a long delay 
in issuing NICARE a contract) and difficulties in agreeing with the Zimbabwe Ministry of Health the 
procedures to be adopted for paying for and controlling local inputs (training, consultancies and top 
management team support).Increased efforts were made to take remedial action two years after the 
project started. 

e — There was a one year delay in employing the financial management adviser and two and a half years in 
arranging the health management adviser. The procedures for identifying individuals for the posts 
proved slow, partly because the advertising was poorly targeted and late and partly due to the time 
taken to obtain the Zimbabwe Government's approval for the proposed consultant and his 
responsibilities. These delays seriously disrupted the rest of the project. 

e Delays in agreeing the mechanics for transferring funds for in-country expenditure resulted in little 
in-country training, local consultancy or support for the top management team (Allocation £435,000; 
Expenditure by November 1994 £204,000). 

e From 1994-95 the Administration have made quarterly advances against expenditure proposals with 
the aim of facilitating the disbursement of funds. The funding arrangement has been adopted by other 
country programmes. 

e There has been considerable difficulty with the procurement of equipment. Hundhill was appointed in 
December 1992, some 15 months after project approval, soon after which they went out of business. 
New procurement agents were not appointed until September 1993 but there was no contract and fees 
were paid on the basis of a memorandum of understanding relating to incomplete Hundhill contracts. 
As a result, the new agents received fees of up to 10% rather than the 5% they had originally bid in 
competition with Hundhill. The Administration considered their performance poor and some of their 
work was Carried out by the finance adviser or his staff. Balfour Williamson consider that they faced a 
number of difficulties in this case which hampered progress. Most of the equipment did not arrive until 
May 1994. (Procurement allocation £231,000; expenditure as at November 1994 £207,000) 

e The management adviser should have identified requirements and developed the management tools; 
instead the finance adviser was employed first and by necessity the finance component drove the 
project 


Training (Allocation £155,000: Expenditure by November 1994 £36,000) 

e The Ministry of Health had difficulty in obtaining access to Administration funds and virtually no 
spending has been incurred by the in-country project activities. The activities of the financial 
management adviser have, as a consequence, been severely restricted. 


Impact 

e — With the exception of the finance component there had been little early project progress. The 
short-term consultancy has been undertaken, as has the link with the UK Health Authority but the 
impact of these is not yet clear. 

e — Expenditure has been slower than estimated - £957,100 was spent by November 1994 against an 
expected £2 million. 

e The cost recovery target was set at Z$60m a year by 1994-95 - 10% of the Ministry of Health’s 
expenditure. Actual data were not available in a convenient form. 


(a) Finance component (Allocation £135,000: Expenditure by November 1994 £133,000) 
There has been substantial progress in producing computerised accounting systems necessary for 
effective management, but more work is needed at the strategic planning and forecasting levels. The 
computer systems have been implemented at the Ministry of Health’s central offices and will gradually be 
cascaded to the provinces and central hospitals. 
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A. Health Management Strengthening 


(b) General management component (Allocation £135,000: Expenditure by November 1994 £78,000) 
There had initially been only limited progress made as the health management adviser was not appointed 
until February 1994. Work has been undertaken by NICARE on training needs assessments, and pilot 
training workshops have been held. Further progress has now been achieved with the production of a 
corporate plan and a paper on a Patients’ Charter, which were additional requirements of this component. 


(c) Management information systems component 

There has been poor progress in agreeing the management information requirement due to differences of 
approach and a lack of agreement on implementation priorities. Some work is being carried out by another 
donor to design an information database. 


(d) Equipment component 
The Ministry of Health and Child Welfare were not satisfied with progress; requirements changed and 
following high level discussions the component was incorporated into another project. 


(e) Supplies component 

Existing manual systems have been standardised and rationalised stores management systems have been 
introduced into two central hospitals which is expected to improve the quality of services provided. Work 
on computerising systems was not started until July 1994 as priority was given to improving the existing 
manual system. 


(f) Public Health component 
An additional component with limited progress and no impact. A draft public health report has been 
produced and a medical audit framework developed. 


(g) Health services planning and management component 
By default this has taken a low priority. The identification of skills requirements has been carried out but the 
department has not yet been strengthened as planned. 


(h) Link with UK Health Authority (Allocation £424 000: Expenditure by November 1994 £262,000) 
UK experts from various health disciplines have proved most useful. This component is seen locally as an 
innovative and potentially useful link, although visits were infrequent and too short. It galvanised local 
action at the time of a visit but the lack of a continued presence has reduced its impact. 


Sustainability 

e Project ownership is not in question. The Ministry of Health have been very supportive but frustrated 
by delays in obtaining technical assistance, supplies and equipment, and in agreeing procedures for 
local expenditure. 

e The Ministry of Health have remained committed to decentralising financial authority and planning. 
They are also committed to producing a more efficient use of resources and providing more effective 
health services. 

e The results of a recent review by consultants from the Institute of Health Sector Development indicate 


that full use is now being made of technical cooperation input. 


3 The main lessons from the management strengthening project are: 


e projects should be as fully designed as possible before approval and the 
project memorandum updated as the project design develops; 
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related preceding projects should be properly evaluated to identify relevant 
lessons; 


appropriate indicators should be established at an early stage and 
incorporated in the project framework; 


effective procedures for disbursing funds should be agreed at the outset; 


target dates should be established for inputs and achievement closely 
monitored; and 


essential technical staff should be appointed promptly. 


B. Emergency and Supplementary Supply of Condoms 


Wider Project - To help reduce fertility and HIV transmission rates by supporting the Government of 


Objectives Zimbabwe Family Planning and AIDS Prevention and Control Programmes 
Immediate - To ensure that the supply of condoms to Zimbabwe Is sufficient to meet demand 
Project - To procure and deliver 66m condoms over an 18 month period 

Objectives - To help the Government of Zimbabwe develop a long-term contraceptive strategy 


- Under the supplementary emergency supply project to procure and deliver 24.54m 
condoms by May 1994 (5m by end of March 1994) 


Project - Provision of 66m condoms to Zimbabwe 

Activities - Technical assistance to developing a long-term strategy for contraceptive supplies 

(1992-94) - Supplementary provision of 24.54m condoms following rejection of the original 
Supplies 

Project Cost Total cost: £1.08m plus £0.5m supplementary supply 


UK expenditure: £0.39m plus £0.5m supplementary supply by November 1994 


Project Crown Agents were employed to procure the original condoms and the International 
Management Planned Parenthood Federation to purchase the supplementary supply. Procurement 


agents fees: £50,000 


Project history 


The requirement for 66m condoms was based on a USAID-funded contraceptive forecast consultancy. 
This projected a 30% increase in 1992 and 15% in 1993, representing 22m and 44m condoms 
respectively. 

The Crown Agents ordered 66m condoms in July 1992 from Dongkuk Techo Rubber Industries, to be 
supplied in 6 consignments between September 1992 and November 1993. 

The first shipment arrived in Harare in December 1992 and the second in February 1993; the third was 
ready for dispatch in July 1993. All the condoms had passed pre-delivery inspection but failed the 
post-delivery tests carried out by the Zimbabwe Regional Drug Control Laboratory. They were 
subjected to extensive further testing which concluded that the manufacture was of poor quality. 

The Administration have paid for the first two consignments but the contracts between the 
Administration, the Crown Agents, Dongkuk and the Government of Zimbabwe have been terminated. 


Supplementary supply of condoms 


Because of the failure to deliver satisfactory supplies, the Administration approved another project to 
provide 24.54m condoms. The lowest quotation was provided by the UN Population Fund but the 
International Planned Parenthood Federation was selected because it could provide immediate 
supplies and was considered to have the required local expertise. In the event the IPPF provided the 
condoms from international supply sources, delivery of which was initially delayed. 
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B. Emergency and Supplementary Supply of Condoms 


Supplementary supply of condoms (contd) 

e Despite early delays in delivering the first few consignments arising from customs clearance and local 
laboratory testing, 8.05m condoms were despatched by air by July 1994. The remaining 16.49m 
condoms were shipped in consignments up to the end of August 1994 


Targets 

e _ To procure and deliver 66m condoms and to provide consultancy assistance to assist the Government 
of Zimbabwe with developing a long-term strategy for contraceptive supplies. 

e No specific targets were set for the wider objective of reducing fertility and HIV transmission rates. 


Monitoring 
e Monitoring was carried out mainly by the Crown Agents and IPPF, who provided periodic progress 
reports. 


e —_ The health and population adviser in Malawi also carried out monitoring visits when the condoms 
failed testing and initiated remedial action in the form of the supplementary supply of condoms project 
in November 1993. 


Impact 

e The immediate objectives were not met. Zimbabwe was initially severely short of condoms, putting at 
risk the objective of fertility and HIV transmission rates being reduced. 

e — Remedial action was taken to provide supplementary supplies, initially airfreighted. 

e The aim of filling the gap in Zimbabwe’s condom stockholding has been partially achieved but at a cost. 

e Despite the failure to deliver the original supply of condoms, Zimbabwe has not run out of stocks - this 
is explained in part by a significant drop in demand in 1992, which followed Zimbabwe's decision to 
introduce charges, and a bureaucratic exemption procedure. IPPF consider that stocks will be 
sufficient for Zimbabwe's 1995 requirement. 

e The potential loss to the Administration was initially put at £300,000 but a negotiated settlement with 
the manufacturers by the Crown Agents has reduced the loss to £85,000. 

e __ There was. a risk that the failed condoms would re-appear elsewhere if Dongkuk took the condoms 
back - following negotiations with the manufacturer the failed condoms were disposed of locally. The 
cost to the Administration for storing the faulty condoms in Zimbabwe and in disposing of them safely 
is estimated at £17,000. 

e The failed 24.54m condoms are being replaced by Dongkuk over a twelve month period commencing 
November 1994. The balance of the 66 million condoms was to be supplied by IPPF by February 1995. 


Sustainability 


e USAID, then UNFPA and now the Administration have supplied virtually all Zimbabwe’s condom 
requirements. 


4 The main lessons from the condom supply projects are: 
e pre-shipment testing must be carried out to the appropriate standards; 


e only agents with the relevant expertise should be appointed as procurement 
agents; 


e close monitoring of a country’s contraceptive requirements is essential; and 


e the long-term sustainability of supplying contraceptives should be assessed. 
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C. Sexual Health project 


Wider Project = - To promote sexual health by reducing the incidence and impact of sexually transmitted 


Objectives infections, including HIV and AIDS 

Immediate - To develop cost effective treatment and prevention policies and protocols, and 

Project disseminate them throughout the health system 

Objectives - To improve the quality of provincial and district treatment and prevention services, with 


particular emphasis on services for women 
- To develop Zimbabwean capacity for procuring condoms and ensure adequate supplies. 
- To ensure efficient distribution and improved access to condoms throughout the 


country 
Project - Establish a National Sexually Transmitted Infections Programme Office — 
Activities - Supply 277.5m condoms and improve the distribution system within Zimbabwe 
(1994-2000) - Establish a capacity within the Zimbabwean National Family Planning Clinic to procure 


condoms and computerise stock management 
- Produce 10 operational and policy research studies 
- Train 3,882 health workers in the prevention and care of sexually transmitted infections 
The project was approved in April 1994 and the Government of Zimbabwe formally 
accepted the grant on 15 September 1994. 


Project Cost Total cost: £57.35m (World Bank £43m, Government of Zimbabwe £5.33m) 


UK contribution: £9.016m (£4.118m capital, £4.898m technical co-operation) by 
UK expenditure: — Nil by November 1994 (1994-95 allocation £293,000) 


Project Implementation is monitored by a joint Programme Management Committee. The 
Management Administration field officer will undertake in-country monitoring. Management and 


monitoring charges: £270,000 


Project History 


This five year project consists of two components: the first to establish a National Sexually Transmitted 
Infections Programme Office, equip a central training and referral centre, conduct research and policy 
development, formulate national and provincial action plans, and provide funds for service 
development equipment, transport and management support; the second to supply 277.5m condoms 
and improve the arrangements for their quality testing, storage and distribution 


Targets 
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The project framework included generally quantifiable indicators of impact, although for some specific 
targets the current level of achievement was not clear. Some of the targets rely on assumptions that 
are not well documented. For example, to hold HIV infection at its current level assumes that the 
project will be implemented and effective instantly, preventing any further transmission and that 
condoms will be used regularly by all groups. Factors external to the project, such as an HIV epidemic 
already in progress, could significantly impede the achievement of the project’s objectives. Funded 
activities are appropriate to the wider objective. 

Operational research within the project and activities under the manage eee strengthening project may 
provide some of the baseline data currently lacking in Zimbabwe. 

The project assumes that the condoms supplied will be regularly used by those people at risk but the 
source and reliability of condom usage rate - one of the main target indicators, is on a small study and 
uncertain. 
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C. Sexual Health project 


Project management 


The project was approved in April 1994 with the aim of starting in July 1994. The project commenced 
in September 1994. 

The Administration’s health and population field manager will be responsible for ensuring that funds 
are used. 

Research proposals will be overseen by a research advisory panel of the management committee. 
Effective implementation depends on the timely local recruitment of the key project members. No 
target date has been set although fund disbursement has been made conditional on the appointment of 
the programme manager. 

The Administration will monitor the Zimbabwe Ministry of Health contract with the National Family 
Planning Council to reimburse the cost of condom handling and distribution - it should ensure full cost 
is recovered. 


Monitoring and project progress 


The Administration will fund six consultancies to develop local capacity and employ a consultant to 
monitor and evaluate the effectiveness of the project. 

In-country monitoring will be undertaken by the Administration’s health and population field manager. 
Joint bi-annual reviews will look at operational matters and consider progress on the achievement of 
objectives. 

Since the project was formally accepted, staff have been appointed to the posts of project manager, 
condom manager and administrative officer. 


Condom supply 


The project will provide Zimbabwe’s main supply of condoms from 1995-96 to 1998-99 - some 
277.9m. However the numbers required, based on World Bank projections, need re-assessing. 

The forward projection for condom demand, a 12-fold increase in 11 years, looks high 

A calculation error over-assessed need by some 8.5m condoms. Using the UN Population Fund’s base 
1993-94 figure the total requirement would be 34.5m less; alternatively, by basing the calculation on 
issues, requirement would be 74.5m less. 


Impact 


The project aims, by 1999, to decrease the number of sexually transmitted infection clients visiting 
public sector clinics from 900,000 to 600,000 and to reduce the number of couples contracting 
infections by 800,000. It will provide 2.77m couple years of protection. 


Sustainability 


The project provides for the Zimbabwe Government to contribute an increasing percentage of the cost 
of condoms (from 10% in year 3 to 30% in year 5). There is a risk that this may not be achieved as 
such a level of contribution has not been attempted in a developing country by any other condom 
project. Similarly, direct procurement by recipient governments has not been attempted before but the 
benefits of the Family Planning Council procuring condoms have not been documented. The Council 
could not demonstrate what immediate benefits they would achieve by purchasing the condoms 
themselves; they agreed that any savings on procurement agents’ fees would not be offset by the 
higher price which they may, as a small purchaser, have to pay. 

The Government of Zimbabwe is fully committed to the project and the associated national AIDS 
programme. 

The post-project recurrent costs of £2.8m a year are likely to cause a problem in sustaining the 
programme but the Administration consider they are affordable within the overall Zimbabwe health 
resources available. 

The cost of providing procurement training and developing a long-term strategy for condom supplies 
is £223,000. 
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5 The main lessons from the Sexual Health project are: 


e external factors which impede a project achieving targets should be fully 
documented; 


e target dates should be set for recruiting key personnel; and 


e the benefits and cost-savings from developing an indigenous capacity to 
procure condoms should be highlighted in project proposals. 
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Appendix 5: Multilateral aid 


Since April 1989 the Administration have provided £135 million to three 
multilateral organisations. The United Nations Population Fund, which received 
£38.5 million, promotes awareness of population issues and facilitates 
international co-operation in population issues, demography and family 
planning. Its 1994 budget was £152 million to which the Administration 
contributed £8.5 million. 


The International Planned Parenthood Federation is a voluntary family health 
care organisation working in 130 countries through its associate members. It 
has received £39 million from the Administration since April 1989. In 
collaboration with governments and non-governmental and community 
organisations it provides reproductive health and family planning services. In 
1994 it received £7.5 million from the Administration towards its annual budget 
of £77 million. 


The World Health Organisation is responsible for international public health 
matters and is funded by 184 countries. In addition to the Uk’s regular budget 
contribution, paid by the Department of Health, the Administration contributed 
£57.5 million over the last five years (£16.7 million during 1993-94). 


Contributions to these organisations account for over half of the 
Administration’s expenditure on reproductive health and other population 
activities. Increasing pressure on the aid budget, caused largely by the growth 
in non-discretionary contributions to the European Community, has led the 
Administration to place increased emphasis on value for money and to 
re-assess and adjust the balance of their multilateral contributions. 


The National Audit Office examined how the Administration monitor value for 
money from their contributions to multilateral organisations. The sources used 
to influence the direction and effectiveness of programmes are shown in 
Figure 5. 


The National Audit Office concluded that: 


e through their population services agreement, the Administration have 
established a comprehensive process of review and evaluation, providing 
management with valuable information on the effectiveness of the funded 
agencies; 

e the Administration consider a broad span of evidence, including evaluations 
performed by other donor countries, to assess the effectiveness of funded 
agencies; and 

e the Administration have been active in seeking the reforms necessary to 


ensure that their increasing contributions to World Health Organisation 
Special Programmes achieve optimum effectiveness and value for money. 
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Figure 5: Sources used to influence the direction and effectiveness of programmes 


Form of monitoring undertaken by ODA 


The United Nations Population Fund 

Participation in consultative meetings 

Chief Health and Population adviser accompanied by 
UK mission at UN 


Representation on UNDP Governing Council 
Chief Health and Population adviser 


Administration's evaluations 

Options reviewed programmes in Ghana and 
Pakistan. 

Tanzania and Nepal programmes also reviewed in 
1994 


Evaluations by other donors 
1993 review of Fund activities carried out jointly by 
three other donors. 


International Planned Parenthood Federation 
Participation in annual donors meetings 
Health and Population Division 


Administration’s evaluations 

Options reviewed programmes in Ghana 

and Pakistan. 

Tanzania and Nepal programmes also reviewed in 
1994 


World Health Organisation 

Participation in Assembly meetings 

Department of Health’s Chief Medical Officer plus 
input from the Administration 


Participation in individual programmes 
Health and Population Division 


External reviews of programmes 
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Results or findings 


Raised concerns over the Fund’s policies and 
priorities and encouraged improved monitoring. 


Influence is limited but the Administration used the 
meetings to play an active and constructively critical 
role. 


The Administration did not receive good value for 
money in either country but it was achievable if 
improvements in local management continued. 
Activities lacked clear strategic objectives and 
quantified targets. The Administration are uncertain 
of the Fund’s commitment to the reforms needed to 
improve effectiveness. 

The findings supported the conclusions reached in 
Administration reviews. They provided additional 
information on the effectiveness of the Fund. 


The Administration played an active and 
constructively critical role at meetings. The 
Federation is now providing more evidence to 
donors on how money is spent and on its impact. 
The Administration obtained good value for money 
from Federation activities in Pakistan but not in 
Ghana. Improvements in planning and local 
management are underway in Ghana to establish 
greater value for money. 


Opportunities to discuss and influence the 
Organisation’s policies. The Administration have 
monthly liaison meetings with the Department of 
Health to put their views on development issues. 
Opportunity to examine the programme’s strategic 
options and ensure their relevance to the 
Administration’s objectives. 

The Organisation’s four largest programmes have 
been reviewed in recent years. Improvements in 
co-ordination between UN agencies have been 
suggested. The Administration have been active in 
encouraging greater strategic guidance to developing 
countries. 
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Appendix 6: Research 
expenditure on health and 
population 


Since 1989-90 the Administration have spent £85 million on health and 
population research, including £55 million on international research and 

£30 million through UK organisations. Until 1991 the Administration had no 
formalised health and population research strategy which led to a lack of clarity 
of requirements and objectives. The priorities now set for research include 
communicable diseases, reproductive health problems, non-communicable 
illnesses and the analysis and development of cost-effective health care. The 
objectives are to establish a social and economic climate for people to choose 
when to have children, to improve the availability, accessibility and quality of 
family planning and health care services, and to control communicable diseases. 


The Administration support a wide variety of health and population research 
and development activities through partnerships with academic groups, 
non-governmental organisations, international bodies and the British Medical 
Research Council. Their research funding concentrates on support for 
developing and applying technologies appropriate to developing countries, and 
research into policies, management procedures and systems for resource 
allocation. The aim is to produce results which enable decision makers to 
choose between competing priorities for expenditure and enable them to 
organise management change. 


The Administration have funded international research into AIDS, human 
reproduction, tropical diseases, diarrhoea, respiratory infections and recently 
tuberculosis. UK-based research comprises work programmes, research under 
the HP-ACORD and a grant-in-aid to the Medical Research Council. 


In 1990 the Administration approved support for 14 five-year research 
programmes at the London School of Hygiene and Tropical Medicine and the 
Liverpool School of Tropical Medicine, at an estimated cost of £16 million. By 
June 1993 £8.7 million, including a 40 per cent contribution to overheads, had 
been spent on these programmes; matching resources were obtained from 
other funding bodies. 


From April 1995 these projects were replaced by eight new research 
programmes. The Administration are defining the topic areas to ensure that the 
work undertaken is directly linked to their priorities. Three negotiated projects 
are to be shared by the two schools and will cover malaria, reproductive health 
and health economics research. The remaining five were subjected to open 
competition. 
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In January 1992 the Administration established a Health and Population 
Advisory Committee on Research and Development to review applications for 
research funding for projects costing up to £100,000 over a three year period. 
By August 1994 32 projects had been approved totalling £5.6 million. 


Since April 1990 the Administration have contributed £8.6 million to the 
Medical Research Council, representing some 16.5 per cent of its costs of 
undertaking relevant research over the last three years. Before 1993 the 
Administration did not specifically focus or direct the Council research they 
funded. In December 1992, however,the Administration established a concordat 
with Council which established a framework for the promotion, funding and 
management of scientific research relevant to the health of developing societies. 
Some 85 per cent of the Council’s 1993-94 £11 million programme is on 
bacterial infections, parasitic infections, reproductive health and HIV/AIDS. 


Monitoring of health and population research has varied. Before the concordat 
with the Medical Research Council monitoring was spasmodic and 
unstructured. The work programmes undertaken at the schools of medicine 
were reviewed in 1991 and 1992 and triennial reviews were undertaken in 
1903: 


The August 1993 triennial review of the London School of Hygiene and Tropical 
Medicine concluded that the nine work programmes were helping to develop 
the School’s institutional and training capacity, fostering institutional links with 
developing country institutions, and stimulating new research. The review 
team, however, considered that the value of the individual programmes was not 
easy to measure in precise terms. 


The October 1993 triennial review of the Liverpool School of Tropical Medicine’s 
work programmes concluded that it had achieved its broad objective of 
developing its institutional capacity. The five programmes were on course to 
strengthen training and institutional links but the review felt that it was too 
early to conclude whether the programme objectives would be achieved. Two of 
the programmes had started two years late and “research” became a key 
objective only later in the programme. 


The triennial reviews have proved effective in influencing the direction of the 
individual programmes: many needed to be more tightly focused on specific 
interventions, cost-effectiveness and affordable health measures. A number of 
the initial objectives attached to the research programmes did not directly 
correlate with the Administration’s own priorities and'they have therefore 
introduced a number of changes, including the need to improve the assessment 
of value for money. 


In May 1994 the Administration appointed a consultant research adviser to 
assist in monitoring research projects. This work aims to ensure that research 
efforts better meet the needs of clients in developing countries and thereby add 
value. Evaluations of the work programme and the initial HP-ACORD projects 
will form part of the consultant’s responsibilities. 
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It is rather early to comment on the results of much of the research. The core 

work programme and the first batch of HP-ACORD three-year research projects 
were completed in March 1995. Of the remaining research projects funded over 
the last five years, some 19 projects costing £2.35 million have been completed. 


Six of the London School of Hygiene and Tropical Medicine programmes have 
had an impact on policy formulation at country or international level and on its 
implementation. Two programmes (tropical disease control and AIDS) are 
showing good progress and are likely to achieve their objectives. One 
programme (tropical health technology) has developed a number of new 
technologies, several of which have been transferred to developing countries. 
The October 1993 review found that the Liverpool School’s five programmes 
have had some impact on health policy practices in developing countries; two 
programmes (reproductive health and primary health care management) have 
demonstrated significant achievements. 


The National Audit Office sought to identify the extent to which the results of 
completed research had been assessed by the Administration but only limited 
information was available. The results of the two cases examined are set out 
below. 


Controlled trial of vitamin A supplementation on child survival in Northern Ghana 


Researchers: London School of Hygiene and Tropical Medicine and Kumasi University, Ghana 

Cost: £1.1m 

Objective: to test whether illness and death are reduced in children given capsules of vitamin A regularly in 
an area of known vitamin A deficiency with few health services. 

Results: there was a 19% reduction in deaths in children receiving vitamin A, and fewer hospital admissions 
and clinic attendances. However, the presence of common illnesses, such as chest infections and diarrhoea, 
were not affected - implying that the vitamin acts by reducing the severity of illnesses in vitamin deficient 
children. 

Action taken: recommendations are that the agricultural and health sector act on these findings in Ghana. 
These include introducing vitamin A supplementation mechanisms in the district. Vitamin A supplies should 
be bought nationally and included on the government's essential vitamin list. 


Evaluation of the risks to human health from parasitic nematodes in treated wastewater 
used for irrigation 


Researchers: University of Leeds, Department of Civil Engineering 

Cost: £115,000 

Objective: to evaluate whether the current World Health Organisation guideline of one viable intestinal 
nematode per litre of treated waste water for irrigation of crops for human consumption is set too low. 
Results: confirmed that the guidelines of one egg will lead to plant contamination with namatode eggs; with 
fresh foodstuffs for human consumption, tolerance of levels higher than the guidelines may risk infecting 
people. 

Action taken: a need for further work on improving the effectiveness of wastewater stablisation ponds and 
monitoring nematode eggs in wastewater. 
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16 Various studies have been undertaken on behalf of the Health and Population 
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Division to assist in predicting the potential impact of different forms of health 
and population interventions. Recent work included assessing the best 
public/private mix for health care in developing countries, comparing different 
types of cost recovery systems, the development of new malaria control 
measures, reviews of the effectiveness of health service interventions, and 
consideration of possible revisions to existing emergency feeding programmes. 
The Administration have recently decided to put more effort into summarising 
the findings of research so that it can be more easily disseminated. 


The Administration need to consider the extent to which their support for 
research informs decision making at local, national and international levels and 
how it leads to better health and access to health care in developing countries. 
To ensure that research contributes to effective health care the results need to 
be disseminated and adopted. The Administration need to ensure that 
researchers specifically consider this aspect in the research design, and they 
should also consider how they and the researchers should take an active part in 
disseminating results and encouraging effective policy formulation. 
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Phase | Sub-centre, 
Orissa, India (Appendix 1A) 








Phase II Sub-centre, 
Orissa, India (Appendix 1A) 


Health education street theatre, 
Orissa, India (Appendix 1A) 








New Ngecha clinic nearing 
completion - Christian Health 
Association of Kenya component of 
Population III project (Appendix 3B) 
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Condom testing, Zimbabwe Regional Drugs Control 
Laboratory (Appendix 4B) 





Ngecha clinic - old (note use of bottles for medicines) 
(Appendix 3B) 
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Reports by the Comptroller 
and Auditor General 
Session 1994-95 





The Comptroller and Auditor General has to date, in Session 1994-95, 
presented to the House of Commons the following reports under Section 9 of the 
National Audit Act, 1983: 


Treasury Management in National Health Service Trusts 


im England oo smckedenns << hebe.cteseocsoeace cgevmeuer cel svses uses ane nce ge seats aan HG@ 
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General Practitioner Pundholding in England -..........22...1.-0:.:.:.-5.402 eee HC 38 
Overseas Development Administration: Management of 
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Department for Education: Management of Office Space ......................:0000e HC 7a 
Crown Office and the Procurator Fiscal Service: Scottish 

Courts Administration Resources in Sheriff Courts.....................c:0000000 HCG te 
Resource Accounting and Budgeting in Government.......................scseceseeee HC ta 
Department of Transport: Sale of DVO. 22)...cc.....-..,0eccs sect HC 128 
sale of Forward Catering Services Limited’, (0.0.2 sa.....2c.22:-s-0 see HC ta) 
Managing to be Independent: Management and Financial 

Control at Colleges in the Further Education Sector..........0.00..cceeceeeeeeeee HG ra 
Severance Payments to Senior Staff in the Publicly Funded 

Education Sector s,,).ccc.segee crs eee HC 202 


Second Sale of the Government’s Debt in British 
Telecommunications and Privatised Electricity 


COMpPanies.....ceskieepeementvesso ese ssa aere ee eee eee  dascensdSisanenecesdeeeeeat aan HC 184 
Entry into the United Kingdom... se cee HC 204 
HM Customs and Excise: Writing Off VAT Arrears ...................0ee HG 2039 
IT Security in Government Departments ...............2-.) ee HC dZae 
Inland Revenue: Market Testing the Information 

Technology :Qiice \...scpccuieite.n>. sacOeaen sscendsadleeeegere:e secs oseeteeee eee HC 245 
Ministry of Defence: The Risk of Fraud in Defence 
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Contracting for Acute Health Care in England ................-..... 0 HC 261 
National Rivers Authority: River Pollution from Farms | 
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Interim Report: PSA Services - The Sale of PSA Projects..............::0c:ce00000+ HC 306 
Commission for the New Towns: Disposal of Land and 

Property Assets cig tisdectel ti es.tsn'snciesd ance Sawkccs desde t cee eee HC 308 
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Department of Employment: Financial Controls in 
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National Health Service Outpatient Services in England 
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Department of Social Security: Purchase of Postal 
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Medical Research Council: Sale of the Mount Vernon Site, 
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Ministry of Defence: Major Projects Report 1994.00.00... ssssessseeessseeceeeees HC 436 
Protecting and Presenting Scotland’s Heritage Properties .......................05. HC 430 
A Review of the Financial Controls over Indirectly 

Funded Operations of the Metropolitan Police Service ....................:00 HC 462 
London Docklands Development Corporation: 
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The Banking Service provided by The Office of 
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Ministry of Defence: Procurement Lessons for the Common New 
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The Meteorological Office Executive Agency: Evaluation of Performance...HC 693 
Planning for Change: National Audit Office Examinations of 
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